+
+ This program is free software: you can redistribute it and/or modify
+ it under the terms of the GNU Affero General Public License as
+ published by the Free Software Foundation, either version 3 of the
+ License, or (at your option) any later version.
+
+ This program is distributed in the hope that it will be useful,
+ but WITHOUT ANY WARRANTY; without even the implied warranty of
+ MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ GNU Affero General Public License for more details.
+
+ You should have received a copy of the GNU Affero General Public License
+ along with this program. If not, see .
+
+Also add information on how to contact you by electronic and paper
+mail.
+
+If your software can interact with users remotely through a computer
+network, you should also make sure that it provides a way for users to
+get its source. For example, if your program is a web application, its
+interface could display a "Source" link that leads users to an archive
+of the code. There are many ways you could offer source, and different
+solutions will be better for different programs; see section 13 for
+the specific requirements.
+
+You should also get your employer (if you work as a programmer) or
+school, if any, to sign a "copyright disclaimer" for the program, if
+necessary. For more information on this, and how to apply and follow
+the GNU AGPL, see .
diff --git a/LICENSE-EUPL b/LICENSE-EUPL
deleted file mode 100644
index 4153cd3..0000000
--- a/LICENSE-EUPL
+++ /dev/null
@@ -1,287 +0,0 @@
- EUROPEAN UNION PUBLIC LICENCE v. 1.2
- EUPL © the European Union 2007, 2016
-
-This European Union Public Licence (the ‘EUPL’) applies to the Work (as defined
-below) which is provided under the terms of this Licence. Any use of the Work,
-other than as authorised under this Licence is prohibited (to the extent such
-use is covered by a right of the copyright holder of the Work).
-
-The Work is provided under the terms of this Licence when the Licensor (as
-defined below) has placed the following notice immediately following the
-copyright notice for the Work:
-
- Licensed under the EUPL
-
-or has expressed by any other means his willingness to license under the EUPL.
-
-1. Definitions
-
-In this Licence, the following terms have the following meaning:
-
-- ‘The Licence’: this Licence.
-
-- ‘The Original Work’: the work or software distributed or communicated by the
- Licensor under this Licence, available as Source Code and also as Executable
- Code as the case may be.
-
-- ‘Derivative Works’: the works or software that could be created by the
- Licensee, based upon the Original Work or modifications thereof. This Licence
- does not define the extent of modification or dependence on the Original Work
- required in order to classify a work as a Derivative Work; this extent is
- determined by copyright law applicable in the country mentioned in Article 15.
-
-- ‘The Work’: the Original Work or its Derivative Works.
-
-- ‘The Source Code’: the human-readable form of the Work which is the most
- convenient for people to study and modify.
-
-- ‘The Executable Code’: any code which has generally been compiled and which is
- meant to be interpreted by a computer as a program.
-
-- ‘The Licensor’: the natural or legal person that distributes or communicates
- the Work under the Licence.
-
-- ‘Contributor(s)’: any natural or legal person who modifies the Work under the
- Licence, or otherwise contributes to the creation of a Derivative Work.
-
-- ‘The Licensee’ or ‘You’: any natural or legal person who makes any usage of
- the Work under the terms of the Licence.
-
-- ‘Distribution’ or ‘Communication’: any act of selling, giving, lending,
- renting, distributing, communicating, transmitting, or otherwise making
- available, online or offline, copies of the Work or providing access to its
- essential functionalities at the disposal of any other natural or legal
- person.
-
-2. Scope of the rights granted by the Licence
-
-The Licensor hereby grants You a worldwide, royalty-free, non-exclusive,
-sublicensable licence to do the following, for the duration of copyright vested
-in the Original Work:
-
-- use the Work in any circumstance and for all usage,
-- reproduce the Work,
-- modify the Work, and make Derivative Works based upon the Work,
-- communicate to the public, including the right to make available or display
- the Work or copies thereof to the public and perform publicly, as the case may
- be, the Work,
-- distribute the Work or copies thereof,
-- lend and rent the Work or copies thereof,
-- sublicense rights in the Work or copies thereof.
-
-Those rights can be exercised on any media, supports and formats, whether now
-known or later invented, as far as the applicable law permits so.
-
-In the countries where moral rights apply, the Licensor waives his right to
-exercise his moral right to the extent allowed by law in order to make effective
-the licence of the economic rights here above listed.
-
-The Licensor grants to the Licensee royalty-free, non-exclusive usage rights to
-any patents held by the Licensor, to the extent necessary to make use of the
-rights granted on the Work under this Licence.
-
-3. Communication of the Source Code
-
-The Licensor may provide the Work either in its Source Code form, or as
-Executable Code. If the Work is provided as Executable Code, the Licensor
-provides in addition a machine-readable copy of the Source Code of the Work
-along with each copy of the Work that the Licensor distributes or indicates, in
-a notice following the copyright notice attached to the Work, a repository where
-the Source Code is easily and freely accessible for as long as the Licensor
-continues to distribute or communicate the Work.
-
-4. Limitations on copyright
-
-Nothing in this Licence is intended to deprive the Licensee of the benefits from
-any exception or limitation to the exclusive rights of the rights owners in the
-Work, of the exhaustion of those rights or of other applicable limitations
-thereto.
-
-5. Obligations of the Licensee
-
-The grant of the rights mentioned above is subject to some restrictions and
-obligations imposed on the Licensee. Those obligations are the following:
-
-Attribution right: The Licensee shall keep intact all copyright, patent or
-trademarks notices and all notices that refer to the Licence and to the
-disclaimer of warranties. The Licensee must include a copy of such notices and a
-copy of the Licence with every copy of the Work he/she distributes or
-communicates. The Licensee must cause any Derivative Work to carry prominent
-notices stating that the Work has been modified and the date of modification.
-
-Copyleft clause: If the Licensee distributes or communicates copies of the
-Original Works or Derivative Works, this Distribution or Communication will be
-done under the terms of this Licence or of a later version of this Licence
-unless the Original Work is expressly distributed only under this version of the
-Licence — for example by communicating ‘EUPL v. 1.2 only’. The Licensee
-(becoming Licensor) cannot offer or impose any additional terms or conditions on
-the Work or Derivative Work that alter or restrict the terms of the Licence.
-
-Compatibility clause: If the Licensee Distributes or Communicates Derivative
-Works or copies thereof based upon both the Work and another work licensed under
-a Compatible Licence, this Distribution or Communication can be done under the
-terms of this Compatible Licence. For the sake of this clause, ‘Compatible
-Licence’ refers to the licences listed in the appendix attached to this Licence.
-Should the Licensee's obligations under the Compatible Licence conflict with
-his/her obligations under this Licence, the obligations of the Compatible
-Licence shall prevail.
-
-Provision of Source Code: When distributing or communicating copies of the Work,
-the Licensee will provide a machine-readable copy of the Source Code or indicate
-a repository where this Source will be easily and freely available for as long
-as the Licensee continues to distribute or communicate the Work.
-
-Legal Protection: This Licence does not grant permission to use the trade names,
-trademarks, service marks, or names of the Licensor, except as required for
-reasonable and customary use in describing the origin of the Work and
-reproducing the content of the copyright notice.
-
-6. Chain of Authorship
-
-The original Licensor warrants that the copyright in the Original Work granted
-hereunder is owned by him/her or licensed to him/her and that he/she has the
-power and authority to grant the Licence.
-
-Each Contributor warrants that the copyright in the modifications he/she brings
-to the Work are owned by him/her or licensed to him/her and that he/she has the
-power and authority to grant the Licence.
-
-Each time You accept the Licence, the original Licensor and subsequent
-Contributors grant You a licence to their contributions to the Work, under the
-terms of this Licence.
-
-7. Disclaimer of Warranty
-
-The Work is a work in progress, which is continuously improved by numerous
-Contributors. It is not a finished work and may therefore contain defects or
-‘bugs’ inherent to this type of development.
-
-For the above reason, the Work is provided under the Licence on an ‘as is’ basis
-and without warranties of any kind concerning the Work, including without
-limitation merchantability, fitness for a particular purpose, absence of defects
-or errors, accuracy, non-infringement of intellectual property rights other than
-copyright as stated in Article 6 of this Licence.
-
-This disclaimer of warranty is an essential part of the Licence and a condition
-for the grant of any rights to the Work.
-
-8. Disclaimer of Liability
-
-Except in the cases of wilful misconduct or damages directly caused to natural
-persons, the Licensor will in no event be liable for any direct or indirect,
-material or moral, damages of any kind, arising out of the Licence or of the use
-of the Work, including without limitation, damages for loss of goodwill, work
-stoppage, computer failure or malfunction, loss of data or any commercial
-damage, even if the Licensor has been advised of the possibility of such damage.
-However, the Licensor will be liable under statutory product liability laws as
-far such laws apply to the Work.
-
-9. Additional agreements
-
-While distributing the Work, You may choose to conclude an additional agreement,
-defining obligations or services consistent with this Licence. However, if
-accepting obligations, You may act only on your own behalf and on your sole
-responsibility, not on behalf of the original Licensor or any other Contributor,
-and only if You agree to indemnify, defend, and hold each Contributor harmless
-for any liability incurred by, or claims asserted against such Contributor by
-the fact You have accepted any warranty or additional liability.
-
-10. Acceptance of the Licence
-
-The provisions of this Licence can be accepted by clicking on an icon ‘I agree’
-placed under the bottom of a window displaying the text of this Licence or by
-affirming consent in any other similar way, in accordance with the rules of
-applicable law. Clicking on that icon indicates your clear and irrevocable
-acceptance of this Licence and all of its terms and conditions.
-
-Similarly, you irrevocably accept this Licence and all of its terms and
-conditions by exercising any rights granted to You by Article 2 of this Licence,
-such as the use of the Work, the creation by You of a Derivative Work or the
-Distribution or Communication by You of the Work or copies thereof.
-
-11. Information to the public
-
-In case of any Distribution or Communication of the Work by means of electronic
-communication by You (for example, by offering to download the Work from a
-remote location) the distribution channel or media (for example, a website) must
-at least provide to the public the information requested by the applicable law
-regarding the Licensor, the Licence and the way it may be accessible, concluded,
-stored and reproduced by the Licensee.
-
-12. Termination of the Licence
-
-The Licence and the rights granted hereunder will terminate automatically upon
-any breach by the Licensee of the terms of the Licence.
-
-Such a termination will not terminate the licences of any person who has
-received the Work from the Licensee under the Licence, provided such persons
-remain in full compliance with the Licence.
-
-13. Miscellaneous
-
-Without prejudice of Article 9 above, the Licence represents the complete
-agreement between the Parties as to the Work.
-
-If any provision of the Licence is invalid or unenforceable under applicable
-law, this will not affect the validity or enforceability of the Licence as a
-whole. Such provision will be construed or reformed so as necessary to make it
-valid and enforceable.
-
-The European Commission may publish other linguistic versions or new versions of
-this Licence or updated versions of the Appendix, so far this is required and
-reasonable, without reducing the scope of the rights granted by the Licence. New
-versions of the Licence will be published with a unique version number.
-
-All linguistic versions of this Licence, approved by the European Commission,
-have identical value. Parties can take advantage of the linguistic version of
-their choice.
-
-14. Jurisdiction
-
-Without prejudice to specific agreement between parties,
-
-- any litigation resulting from the interpretation of this License, arising
- between the European Union institutions, bodies, offices or agencies, as a
- Licensor, and any Licensee, will be subject to the jurisdiction of the Court
- of Justice of the European Union, as laid down in article 272 of the Treaty on
- the Functioning of the European Union,
-
-- any litigation arising between other parties and resulting from the
- interpretation of this License, will be subject to the exclusive jurisdiction
- of the competent court where the Licensor resides or conducts its primary
- business.
-
-15. Applicable Law
-
-Without prejudice to specific agreement between parties,
-
-- this Licence shall be governed by the law of the European Union Member State
- where the Licensor has his seat, resides or has his registered office,
-
-- this licence shall be governed by Belgian law if the Licensor has no seat,
- residence or registered office inside a European Union Member State.
-
-Appendix
-
-‘Compatible Licences’ according to Article 5 EUPL are:
-
-- GNU General Public License (GPL) v. 2, v. 3
-- GNU Affero General Public License (AGPL) v. 3
-- Open Software License (OSL) v. 2.1, v. 3.0
-- Eclipse Public License (EPL) v. 1.0
-- CeCILL v. 2.0, v. 2.1
-- Mozilla Public Licence (MPL) v. 2
-- GNU Lesser General Public Licence (LGPL) v. 2.1, v. 3
-- Creative Commons Attribution-ShareAlike v. 3.0 Unported (CC BY-SA 3.0) for
- works other than software
-- European Union Public Licence (EUPL) v. 1.1, v. 1.2
-- Québec Free and Open-Source Licence — Reciprocity (LiLiQ-R) or Strong
- Reciprocity (LiLiQ-R+).
-
-The European Commission may update this Appendix to later versions of the above
-licences without producing a new version of the EUPL, as long as they provide
-the rights granted in Article 2 of this Licence and protect the covered Source
-Code from exclusive appropriation.
-
-All other changes or additions to this Appendix require the production of a new
-EUPL version.
diff --git a/README.md b/README.md
index 5b8b09b..37b5961 100644
--- a/README.md
+++ b/README.md
@@ -48,6 +48,6 @@ Please don't feel obligated to donate at all.
## License
-This is released under the EUPL 1.2 license, and based on Lynnesbian's fork which is under the MPL 2.0 license. See LICENSE-EUPL and LICENSE-MPL for details.
+This is released under the AGPLv3 (only) license, and based on Lynnesbian's fork which is under the MPL 2.0 license. See LICENSE-AGPL.md and LICENSE-MPL for details.
**This means you must publish the source code of any ebooks bot you make with this.** A link back to this repository on your bot's profile page or profile metadata will suffice. If you make changes to the code you need to link to your fork/repo instead
diff --git a/functions.py b/functions.py
deleted file mode 100755
index 46d3b3f..0000000
--- a/functions.py
+++ /dev/null
@@ -1,134 +0,0 @@
-# SPDX-License-Identifier: EUPL-1.2
-
-import re
-import os
-import html
-import json
-import shutil
-import sqlite3
-import argparse
-import markovify
-import json5 as json
-import multiprocessing
-from random import randint
-from bs4 import BeautifulSoup
-
-def arg_parser_factory(*, description):
- parser = argparse.ArgumentParser(description=description)
- parser.add_argument(
- '-c', '--cfg', dest='cfg', default='config.json', nargs='?',
- help='Specify a custom location for the config file.'
- )
- return parser
-
-def parse_args(*, description):
- return arg_parser_factory(description=description).parse_args()
-
-def load_config(cfg_path):
- with open('config.defaults.json') as f:
- cfg = json.load(f)
-
- with open(cfg_path) as f:
- cfg.update(json.load(f))
-
- if not cfg['site'].startswith('https://') and not cfg['site'].startswith('http://'):
- print("Site must begin with 'https://' or 'http://'. Value '{0}' is invalid - try 'https://{0}' instead.".format(cfg['site']), file=sys.stderr)
- sys.exit(1)
-
- if 'access_token' not in cfg:
- print('No authentication info', file=sys.stderr)
- print('Get a client id, client secret, and access token here: https://tinysubversions.com/notes/mastodon-bot/', file=sys.stderr)
- print('Then put `access_token` in your config file.', file=sys.stderr)
- sys.exit(1)
-
- return cfg
-
-def make_sentence(output, cfg):
- class nlt_fixed(markovify.NewlineText): # modified version of NewlineText that never rejects sentences
- def test_sentence_input(self, sentence):
- return True # all sentences are valid <3
-
- shutil.copyfile("toots.db", "toots-copy.db") # create a copy of the database because reply.py will be using the main one
- db = sqlite3.connect("toots-copy.db")
- db.text_factory = str
- c = db.cursor()
- if cfg['learn_from_cw']:
- ignored_cws_query_params = "(" + ",".join("?" * len(cfg["ignored_cws"])) + ")"
- toots = c.execute(f"SELECT content FROM `toots` WHERE cw IS NULL OR CW NOT IN {ignored_cws_query_params} ORDER BY RANDOM() LIMIT 10000", cfg["ignored_cws"]).fetchall()
- else:
- toots = c.execute("SELECT content FROM `toots` WHERE cw IS NULL ORDER BY RANDOM() LIMIT 10000").fetchall()
-
- if len(toots) == 0:
- output.send("Database is empty! Try running main.py.")
- return
-
- nlt = markovify.NewlineText if cfg['overlap_ratio_enabled'] else nlt_fixed
-
- model = nlt(
- "\n".join([toot[0] for toot in toots])
- )
-
- db.close()
- os.remove("toots-copy.db")
-
- if cfg['limit_length']:
- sentence_len = randint(cfg['length_lower_limit'], cfg['length_upper_limit'])
-
- sentence = None
- tries = 0
- while sentence is None and tries < 10:
- sentence = model.make_short_sentence(
- max_chars=500,
- tries=10000,
- max_overlap_ratio=cfg['overlap_ratio'] if cfg['overlap_ratio_enabled'] else 0.7,
- max_words=sentence_len if cfg['limit_length'] else None
- )
- tries = tries + 1
-
- # optionally remove mentions
- if cfg['mention_handling'] == 1:
- sentence = re.sub(r"^\S*@\u200B\S*\s?", "", sentence)
- elif cfg['mention_handling'] == 0:
- sentence = re.sub(r"\S*@\u200B\S*\s?", "", sentence)
-
- output.send(sentence)
-
-def make_toot(cfg):
- toot = None
- pin, pout = multiprocessing.Pipe(False)
- p = multiprocessing.Process(target=make_sentence, args=[pout, cfg])
- p.start()
- p.join(5) # wait 5 seconds to get something
- if p.is_alive(): # if it's still trying to make a toot after 5 seconds
- p.terminate()
- p.join()
- else:
- toot = pin.recv()
-
- if toot is None:
- toot = 'Toot generation failed! Contact io@csdisaster.club for assistance.'
- return toot
-
-
-def extract_toot(toot):
- toot = html.unescape(toot) # convert HTML escape codes to text
- soup = BeautifulSoup(toot, "html.parser")
- for lb in soup.select("br"): # replace
with linebreak
- lb.name = "\n"
-
- for p in soup.select("p"): # ditto for
- p.name = "\n"
-
- for ht in soup.select("a.hashtag"): # convert hashtags from links to text
- ht.unwrap()
-
- for link in soup.select("a"): # convert with linebreak
+ el.replace_with('\n')
+
+ for ht in soup.select("a.hashtag, a.mention"): # convert hashtags and mentions from links to text
+ ht.unwrap()
+
+ for link in soup.select("a"): # convert