It’s spoiler time for the upcoming Magic: the Gathering set. During this time a lot of images get posted to reddit. Unfortunately a lot of people can’t see those images because magic.wizards.com is blocked for them. Imgur however, often is not. I happen to be one of those people. Whenever I would see a post about a new card I would click it only to see that the content is blocked. Then I’d check the comments to see if anybody re-posted it on imgur, most of the time nobody had.
I figured this would be a good opportunity to try to create a bot that does just that!
There are handful of requirements that went into creating this bot:
- Create an account for the bot and register an application for it.
- Use a library to interact with the reddit API.
- Use a library to interact with the imgur API.
- A way to keep track of which posts the bot has replied to
Part 1 is easy. I had previously made an account and registered an application in my Getting reddit front page links on Android post.
PRAW is a great library for interacting with reddit. Likewise imgurpython is a great library for interacting with imgur. They both have tons of features that I only scratched the surface of while making this bot. They’re both easy to install with pip.
The OAuth authentication process was new to me. I had initially thought it would be as easy as providing a username/password combo, but it actually appears to work quite differently. You first give the PRAW your application’s info: client_id and client_secret. You then have the user use the actual reddit site to give your application access to their account. That request will return a request code. This request code is a one-time use code to get the information of the user so that you can interact with reddit on their behalf.
That process took a couple of different Python scripts.
The first one was to get the access code for the information of the bot’s reddit account:
import praw from bot_config import * r = praw.Reddit('OAuth getter for wizards_to_imgur script') r.set_oauth_app_info(client_id=reddit_client_id, client_secret=reddit_client_secret, redirect_uri=reddit_redirect_uri) # get_authorize_url takes 3 params: # 1) state: a string of your choice that represents this client # 2) scope: the reddit scope(s) we ask permission for # 3) refreshable: determines whether we can refresh the access_token which will allow permanent access scope = 'identity read submit' url = r.get_authorize_url(user_agent, scope, True) import webbrowser webbrowser.open(url)
bot_config.py contains my info for reddit_client_id
, reddit_client_secret
, reddit_redirect_uri
, and user_agent
. The redirect URI was just http://127.0.0.1/
Once I ran that I was sent to this screen:
After clicking “Allow” I was redirected to a URL that looked like this: http://127.0.0.1/?state=theStateIPutIn&code=MyAccessCode
(The actual values of state and code were different of course)
Once I got the code I ran a second script to get the access information:
import praw from bot_config import * access_code = '<access code here>' red = praw.Reddit(user_agent=user_agent) red.set_oauth_app_info(client_id=reddit_client_id, client_secret=reddit_client_secret, redirect_uri=reddit_redirect_uri) access_info = red.get_access_information(access_code) with open('access_information.txt', 'w') as file: file.write(str(access_info))
Since the code is a one-time use deal, and since I wanted to be able to turn the bot on/off at will that means I had to save the access information.
The access information is a dictionary which is just written to a file called access_information.txt
. The main bot script will read in the info and convert it back into a dictionary.
Here’s the main bot script:
import praw import re import os import ast from bot_config import * from imgurpython import ImgurClient replies_file_name = 'posts_replied_to.txt' if not os.path.isfile(replies_file_name): posts_replied_to = [] else: with open(replies_file_name, 'r') as file: id_list = file.read() id_list = id_list.split('\n') posts_replied_to = [id for id in id_list if id is not None and id is not ''] newly_replied_to = [] reddit = praw.Reddit(user_agent=user_agent) reddit.set_oauth_app_info(client_id=reddit_client_id, client_secret=reddit_client_secret, redirect_uri=reddit_redirect_uri) with open('access_information.txt', 'r') as rf: info = rf.read() access_information = ast.literal_eval(info) reddit.set_access_credentials(**access_information) reddit.refresh_access_information(access_information['refresh_token']) subreddit = reddit.get_subreddit('magictcg') url_pattern = r'^https?:\/\/(www\.)?media\.wizards\.com\/.+\.(png|jpg)$' imgur_client = ImgurClient(imgur_client_id, imgur_client_secret) submissions = subreddit.get_new(limit=25) for submission in submissions: s = '{0}^ ({1}) {2} [{3}]'.format(submission.score, submission.id, submission.title, submission.url) match = re.search(url_pattern, submission.url) if match and submission.id not in posts_replied_to: image = imgur_client.upload_from_url(submission.url, config=None, anon=True) comment = '[{0}]({0})\n\n' \ 'This is bot rehosts images that might otherwise be blocked for people. ' \ 'For issues contact <my user account>' comment = comment.format(image['link']) submission.add_comment(comment) newly_replied_to.append(submission.id) print(s) with open(replies_file_name, 'a') as file: for post_id in newly_replied_to: file.write(post_id + '\n')
Notice at the beginning the contents of posts_replied_to.txt
are read in, and at the end the IDs of any posts replied to are appended. That takes care of requirement 4.
The reddit.refresh_access_information()
call is necessary to be able to add comments to posts. You need to do that every 60 minutes to keep that ability up. I figured it would just be easier to do it every time the script is run. I’m not sure if that’ll cause problems but it seemd to work okay in testing.
The bot tries to match a regular expression against the post’s link to see if it’s an image on media.wizards.com. In theory I could make a regular expression for each site that I’d like to have this bot work for. The catch is that the link must be directly to a picture, ie: the URL must end in .jpg or .png.
Here’s the script to run the bot (runbot.sh):
#!/bin/bash while : do python3 magic_bot.py sleep 1m done
I used an infinite loop in a bash script so I could start it and open it back up with the screen
command. Alternatively I could have put it in the crontab as well. That may end up being the method of choice if I end up creating multiple bots
There’s lots of room for improvement and added features on this bot, however I’d like to see how it’s accepted before I work further on it and add more features.
Final note: it had somehow slipped my mind to ask the mods of the subreddit before I deployed the bot. I asked the users of the subreddit in an open topic for their opinion, but not the mods. When I first activated my bot it was immediately shadowbanned. Presumably for being a new account that made too many posts in too short of a time-frame. When that happened I messaged the mods and explained myself, and the bot is currently running on a trial basis as of this post. Lesson learned, always ask the mods first!