Mimsy: Hacks

Flashcards for iPad using Pythonista—Wednesday, November 21st, 2018

Since I’ve started using Duolingo, I occasionally save a phrase that I think might be worth remembering. I save it very simply: by taking a screenshot. I do this because I think it will be helpful, for the learning process, to memorize some phrases as well as run through the app normally.

What I’ve been doing is occasionally going into the Photos app on the iPad and scanning through the “flashcards” I have thus created. As the number of phrases increases, it has occurred to me that a random flashcard app might be useful.

This is, of course, easy to create in Pythonista.

In the interface builder, two elements are required, a button to show the image and a button to choose the album to get the images from.

button namex, ysizeflexborderradiusborder colortitleaction
flasher6, 12756, 880WH20#6e6e6eTap To StartshowCard
albumChooser166, 900440, 30WLRT18#6e6e6eNo Albums FoundchooseAlbum

The overall view I’ve set to 768x960. The two titles—Tap To Start and No Albums Found—are defaults; the image where the flashcards go will initially not have an image in it and will say “Tap To Start”. Once an image is chosen, that title will go away, by setting its title to an empty string.

PhraseCard view Inspector

The overall view.

PhraseCard flasher Inspector

The button that holds the displayed image.

PhraseCard chooser button inspector

The button that lets you choose which album to use.

The very first function that it runs is:

Preflighting blog comments in the Pythonista share screen—Wednesday, August 22nd, 2018
Pythonista share screen

Pythonista’s share screen lets you run any script you have stored locally; you can also bring up the console, and add commonly-used scripts to a shortcuts area.

The more I use it, the more useful Pythonista becomes at customizing my iPad and iPhone experience. I don’t do a whole lot of commenting on other people’s blogs. One I do comment on occasionally is the Ace of Spades HQ, and it has a somewhat… fragile… 1 commenting system. Specifically, (a) there’s a disconnect with how it handles UTF-8 characters and what it tells browsers it can handle, which means any diacriticals or smart quotes will cause it to fail spectacularly; and (b) it uses a primitive BBCode-like means of italicizing that makes it very possible to “end up in the barrel” by not closing your tags. Not closing your tags means every succeeding comment will be italicized (or bolded, or underlined, or so on) too.

I do a lot of my commenting on the iPhone or iPad, where it is very easy to get screwed over by either autocorrect or fat fingers.

It occurred to me yesterday that this was a great opportunity to start using Pythonista’s sharing extension. You can turn on Pythonista in the sharing window, and it will allow you to run scripts2. Normally, this means running scripts on a text document that you’re “sharing” with Pythonista, but it also has access to the clipboard and whatever text is currently selected.

Having written a script to ensure that I don’t post non-ASCII characters and that I don’t forget to close my brackets, I realized I could make formatting a lot easier by using a simple Markdown converter.

I’m stuck in a computer program; now what?—Sunday, April 1st, 2018
Nasa IMAGE payload

IMAGE payload deck during integration of sleeper Skynet brain?

Please help me i’m sucked in a computer program please please help me

Unfortunately, once you get sucked into a computer program, it’s difficult if not impossible for someone outside the program to help you escape. You’re probably going to have to overcome several trials to find the means of exiting the program. Pay special attention to I/O devices: one of them is the means of outputting yourself back into the real world.

If you have recently experienced severe trauma, emotional or physical, in the real world, you will need to address those issues before you leave the computer program. Somewhere in your current virtual reality are digital analogues to whatever has caused this trauma, for example, your parents, your school counselor, the colleague who shot you with an organic pistol, or the administrators who dismissed you from ENCOM.

If the central core of the hardware your program runs on is based on biological life, you will probably also have to help that composite creation overcome, or at least confront, its own trauma before you can leave.

If you find that you have overcome your emotional trauma by forming a special relationship with a digital being who is not an analog to any outside entity, you are very close to exiting the program. When he or she breaks up with you—or even better, sacrifices themselves to save your life—look closely for an I/O port that will allow you to exit the program or hardware.

The port you’re looking for can go under just about any name, but if you run across a port whose name is prefixed or otherwise modified by another language’s word for “life”, you’ve probably found your exit.

If someone inside the computer tries to convince you that life is better inside than in the real world, understand that there is someone on the outside who is desperately waiting for you to return. It’s possible you don’t even know who they are, but they do exist, and when you exit the program life will be better than it was before you entered the program.

This is true even when you have lived all of your life inside the program.

The more dangerous scenario is that you are inside a program whose boundaries are not limited to the earth; perhaps you’ve been sucked into a spaceship’s computer, or into an interplanetary or interstellar network. In the latter case, it is probably in your best interest to find your way back to your home planet’s I/O ports before you escape.

Renumber selected lines of text—Friday, December 16th, 2016

I use this script extensively with Automator as a service in the Services menu, to, as the title says, renumber lines. It’s extremely simple, and that means it can renumber across multiple lines:

1.	Do this.
	Explanation of doing this.
1.	Do that.
	Explanation of doing that.
1.	And maybe do this other thing.
	Explanation of why I might want to.

The script looks for numbers at the beginning of lines, and that’s it. It allows for tabs and spaces preceding the number, and remembers the first one when it modifies the counter. On subsequent lines, it only modifies the line if the leader matches and is followed by a number.

[toggle code]

  • #!/usr/bin/perl
  • $counter = 0;
  • while (<>) {
    • if (/^([\t ]*)([0-9]+)/) {
      • if ($counter == 0) {
        • $leader = $1;
        • $counter = $2;
      • }
      • $counter++ if s/^$leader[0-9]+/$leader$counter/;
    • }
    • print;
  • }

The first time it encounters a number, it uses that number as the starting point. Subsequent numbers will increment from this number.

It handles only flat lists, because that’s all I’ve ever needed. However, because it checks the spaces and tabs in front of the first selected number, it will not harm indented lists.

I use it mainly (a) when I insert a new item among existing items, and (b) when I re-order items, putting the numbers out of whack.

The easiest way to set it up is using Automator. From the Utilities library, choose Run Shell Script and put the full path to the script into the text box, as pictured. Remember to ensure that the script gets its input from “stdin”. You may also have to check the box that reads “Output replaces selected text”.

1.	Do this.
	Explanation of doing this.
2.	Do that.
	Explanation of doing that.
3.	And maybe do this other thing.
	Explanation of why I might want to.

Whatever you name the Service when you File:Save will be the name of the Service in the Services menu.

Once the script is ready, all I need to do is select the lines I want renumbered, and choose (in my case) Renumber Text Lines from the Services menu of the app.

Calculate poster pixel sizes from an existing image file—Friday, December 2nd, 2016
Three hallway posters

Three 14-inch by 10-inch posters.

I use Zazzle a lot for posters around the house, usually sizing them to fit either a specific space or even a specific frame that I happen to have. Over the Thanksgiving sale weekend, they had a 65% off sale on all posters, including wrapped canvases, so I loaded up on a bunch of posters I’d been wanting to get. I love the big discounts because they let me experiment without wasting much money. At that discount, the most expensive paper poster I ordered was $8.80, and that was for a custom-sized 37½-inch by 25½-inch poster. Even the 40-inch by 20-inch wrapped canvas was only $54.25.

When uploading an image, I like to upload the image already cropped to the appropriate size—either removing the height or the width, depending on which needs to be modified. It’s easy enough to do on a calculator, but also easy enough to make stupid mistakes, such as forgetting that for a wrapped canvas, the wrapped portion needs to be doubled to account for the correct width and height. I also tend to memorize the standard numbers, and forget that I’ve already cropped an image to remove parts I don’t want on the poster.

For repetitive mathematical tasks that a human such as myself can easily forget a step, I write a script, and that’s what I did.

[toggle code]

  • #!/usr/bin/python
  • import argparse
  • parser = argparse.ArgumentParser(description='Calculate poster pixel sizes for existing image size')
  • parser.add_argument('dimensions', help='width and height of poster', nargs=2, type=float)
  • parser.add_argument('pixels', help='width and height in pixels of image', nargs='*', type=int, default=[4032, 3024])
  • parser.add_argument('--canvas', help='size of canvas sides', type=float, default=0)
  • args = parser.parse_args()
  • if len(args.pixels) != 2:
    • print 'Pixels must be two numbers, width and then height.'
    • parser.print_help()
  • posterWidth = args.dimensions[0]
  • posterHeight = args.dimensions[1]
  • width = posterWidth+args.canvas*2
  • height = posterHeight+args.canvas*2
  • pixelWidth = args.pixels[0]
  • pixelHeight = args.pixels[1]
  • print 'For a', posterWidth, 'by', posterHeight, 'poster, this image should be:',
  • #try adjusting the height
  • newHeight = int(round(pixelWidth*height/width))
  • if newHeight <= pixelHeight:
    • print pixelWidth, 'x', newHeight
Bathroom canvas

A ¾-inch wrap canvas, 10-inches by 14-inches.

The script requires providing the poster size as width and then height. If I don’t provide an image size following the poster size, it assumes that I used my dedicated camera and that I haven’t cropped it in any way. That’s the most common image size I use for posters, 4032 by 3024. You’ll want to replace those numbers with the most common image sizes you use.

Unfortunately, Python’s argparse doesn’t provide an easy way to specify that if an optional positional argument is provided, there must be exactly two of them, so the script just checks for that in code.

Then, the script determines what the height of the image needs to be if the height is adjusted. If the new height is possible—that is, if the new height is less than the existing height—then that’s the answer. If the new height is greater than the image’s existing height that’s not doable, so it calculates what the new width will need to be.

Primitive data transfer script for the Model 100/200—Friday, May 27th, 2016

As I’ve mentioned a couple of times now, I have recently acquired a TRS-80 Model 100 and a Tandy Model 200, two very early and very impressive laptops.

Since these laptops predated inexpensive memory both in the form of RAM and permanent storage, there isn’t any room on them to store very many BASIC programs. Further, their primary purpose is writing, and it would be nice to get the writing off of the laptops and onto, say, this blog. The laptops both have several communications options, but as they are options from 1983/1984, none of them are Internet, Bluetooth, or WiFi.

I ordered a RS-232 to USB adapter and delved into Python’s serial port extension module, pyserial.

This stuff is ancient; so much about communications nowadays is handled by software that I’d completely forgotten the need for a null modem adapter to switch the transmit and receive lines in RS-232 when connecting two computers directly to each other. Further, it isn’t used enough to justify drivers that don’t crash your computer. After running these tests on my iMac using the Prolific drivers, I now habitually unplug the serial adapter and reboot the computer. If I don’t the computer will reboot itself within a few hours. And this is the most commonly recommended product/driver combination among people who still use RS232.

That working, however, what I needed was very simple. The Model 100/200 does not have any fancy terminal software with downloading functions that automatically recognize filenames and download to the appropriate place. “Download” on these laptops is no more than capturing the incoming data to a file you name, and “upload” is no more than sending out the file you want to upload. I vaguely remember, in the early days of Bulletin Board Services, special codes for noting when a file transfer was about to start so that terminal software would know; these laptops predate that.

And all I really need is to be able to transfer text files back and forth. I chose to make the file transfer location be under my Dropbox folder; I named the folder “TRS80”. This way, I can also put things into it and take things out of it on Editorial or Textastic on the iPad.

Goodreads: What books did I read last week and last month?—Friday, February 26th, 2016

I started using Goodreads in 2014, and it’s a nice way of tracking what books I read and what I thought about them. One thing I have definitely noticed missing, however, is an advanced search. Every once in a while I want to see what and how many books I read in the last week, or the last month, or the last x days, and there is no such search. You have to count them up yourself.

At the end of the year, I wanted to find all of the books from 2015 that I’d rated at 5. The only way I could find to do this was to eyeball the list. An advanced search would have made this easy.

In my Django database of books purchased, this sort of data is easy to drill into. It’s very easy to see how many books I purchased in February 2015, for example. And since one of my New Year’s Resolutions is that I am going to read more than I buy, comparing books purchased to books read is important!

However, while it does not provide an advanced search Goodreads does offer data export so that you can save your data to your hard drive. If you use Goodreads extensively it’s a good idea to make regular backups. The export creates a CSV, or comma-separated file, of your books, ratings, reviews, pretty much everything associated with each book you read. This allows us to make an advanced search of our own.

I chose Python because I’m familiar with it, and because it has both a CSV module and an in-memory SQL module (based on SQLite 3) built in.

The drawback to the CSV module is that it is relatively old and not unicode aware. In its defense it assumes everything is utf–8, but it doesn’t mark it that way so that the rest of Python knows. Goodreads, fortunately, provides its CSV file as utf–8. It’s not too hard to make a Python generator that will return unicode/UTF8 values when importing from the Goodreads csv file.

One more trick is that the csv reader only knows of one type of value, the string. But if we want last week’s data, we need to be able to search on a date. So, riffing on LMatter’s stack overflow code, I made a UnicodeDictReader that also converts Goodreads’s Date Read to a Python date.

This part of the script then loops through every line in the csv file and inserts the relevant data into an on-the-fly sqlite3 database.

Converting an existing Django model to Django-MPTT—Friday, February 12th, 2016

For a long time I’ve been painfully aware of a glaring inefficiency in my custom blog software built off of Django. A lot of what it does relies on the tree model: folders within folders. Finding all the front-page articles for my main blog means getting all the children of each category, and then any descendants of those children, and so on. Previewing a new blog article can take a minute or more, mainly because of the recursive nature of building the sidebar. There are only seven articles on the front page, but they require looking through a tree with 1,505 articles spread over five levels.

I’ve been aware of various solutions for turning Django into more of a tree-hugger, but they’ve tended either to not work well with an existing installation, or if they did, their documentation hid that. For example, django-mptt until a few years ago hid the very important .rebuild method, which is necessary for building the tree data for an existing dataset.

Having just finished the basic installation, however, django-mptt is very helpful. On my very simple, not very tree-like blog, the Walkerville Weekly Reader, publish time dropped by around 13% to 21%; since it only took about two minutes anyway, that’s not necessarily a big deal. But Mimsy Were the Borogoves, which I’ve been running since the nineties, has a lot more articles and a much more complex structure. Publish time dropped from about 30 minutes to about 10 minutes. That’s worth the installation troubles.

Converting my model to use django-mptt

I installed django-mptt by downloading version 0.8.0, unpacking it, and running setup, but, despite the documentation claiming that it’s okay with Django 1.8.x, it performed an automatic upgrade to Django 1.9.1. I had to downgrade again using:

  • sudo pip uninstall django
  • sudo pip install django==1.8.8

I’m using django-ajax-selects version 1.3.6, which doesn’t work with Django 1.9.x. While the 1.4 version is compatible with Django 1.9, the 1.4 version doesn’t work on inlines, and inlines are the main reason I use ajax-selects. I have a list of 9,000 URLs and 14,000 pages that can be attached via inline to any page, not to mention the keywords and authors and so forth. Without ajax-select, it takes forever for browsers to render pages because those become pull-down menus.

Converting to django-mptt is very easy. I added this to the top of my models.py:

  • from mptt.models import MPTTModel, TreeForeignKey, TreeManager

I had to include TreeManager because I have a custom manager. If you use the standard manager, you shouldn’t need it.

[toggle code]

  • class PageManager(models.Manager):
  • class Page(models.Model):
    • parent = models.ForeignKey(‘self’, blank=True, null=True)

Older posts.