Mimsy Were the Borogoves

Timeout class with retry in Python—Wednesday, April 16th, 2014

In Paramiko, the SSHClient’s connect method has a timeout parameter, but it rarely causes a timeout in some common instances. Since moving from San Diego’s Cox Cable to Round Rock’s Time-Warner, I’ve been seeing stuck connections much more often.

The fix appears to be to use signals: set an alarm before running the line/function that might get stuck, and then remove the alarm afterward. If the alarm has time to go off, it will generate an exception that can then be handled.

Since the inability to connect or fail to connect appears to be extremely random, I decided to combine it with the option to retry the connection:

[toggle code]

  • import signal
  • class TimeoutError(Exception):
    • pass
  • #Paramiko timeout does not work often, if at all
  • # So create a timeout class
  • class Timer(object):
    • def __init__(self, function=None, seconds=30, tries=3, errorMessage='Timeout'):
      • self.seconds = seconds
      • self.tryLimit = tries
      • self.tries = 1
      • self.function = function
      • self.errorMessage = errorMessage
    • def act(self):
      • signal.signal(signal.SIGALRM, self.handleTimeout)
      • signal.alarm(self.seconds)
      • self.function()
      • signal.alarm(0)
    • def handleTimeout(self, signum, frame):
      • if self.tries >= self.tryLimit:
        • raise TimeoutError(self.errorMessage)
      • else:
        • print 'Timed out on try', self.tries, self.errorMessage
        • self.tries = self.tries + 1
        • self.act()
        • print 'Succeeded on try', self.tries

The first class, a subclass of Exception, doesn’t do anything except give us an appropriately-named exception. The second class will raise that exception if the passed function does not complete before the given number of seconds.

First, instantiate the timer; then run the “act” method. If it times out, it will (by default) try again twice; that is, it will try three times. The function is retried simply by calling the act method again.

For example:

Updated Inkscape extension—Saturday, March 15th, 2014

I noticed on the Inkscape forum that a couple of parts of this tutorial are out of date.

First, the line that includes “/Applications/Inkscape.app/Contents/Resources/extensions” is no longer necessary. The path is now known to all extensions, and by removing this line you can make your extensions more portable. For one thing, you no longer have to worry about the location of the Inkscape app.

Second, the .inx files are now full-fledged XML. This means that they need a real declaration at the top. Replace the “inkscape-extension” root element with:

  • <?xml version="1.0" encoding="UTF-8"?>
  • <inkscape-extension xmlns="http://www.inkscape.org/namespace/inkscape/extension">

This makes the full code:

[toggle code]

  • #!/usr/bin/python
  • import copy
  • import inkex, simpletransform
  • class DuplicateMultiple(inkex.Effect):
    • def __init__(self):
      • inkex.Effect.__init__(self)
      • self.OptionParser.add_option('-n', '--number-of-copies', action='store', type='int', dest='number', default=2, help='How many copies would you like?')
      • self.OptionParser.add_option('-x', '--horizontal', action='store', type='float', dest='horizontal', default=0, help='Horizontal distance?')
      • self.OptionParser.add_option('-y', '--vertical', action='store', type='float', dest='vertical', default=0, help='Vertical distance?')
    • def effect(self):
      • transformation = 'translate(' + str(self.options.horizontal) + ', ' + str(self.options.vertical) + ')'
      • transform = simpletransform.parseTransform(transformation)
      • if self.selected:
        • for id, node in self.selected.iteritems():
          • counter = self.options.number
          • while counter > 0:
            • newNode = copy.deepcopy(node)
            • #newNode.set('style', 'fill:none;stroke:#ff0000;stroke-width:2px;stroke-linecap:butt;stroke-linejoin:miter;stroke-opacity:1')
            • simpletransform.applyTransformToNode(transform, newNode)
            • self.current_layer.append(newNode)
            • counter = counter - 1
            • node = newNode
      • #inkex.debug(newNode.attrib)
  • effect = DuplicateMultiple()
  • effect.affect()

And the full .inx file:

Pipe viewer command-line progress report—Wednesday, November 6th, 2013

I’m sitting here waiting for a two and a half gigabyte MySQL dump file to make its way to the Amazon Cloud and basically the only way I have to monitor its progress is opening up a Sequel Pro window and wait for new databases to show up.

I thought I’d do a search on MySQL dump progress reports, and I found a neat little utility called pipe viewer that really ought to be standard on Mac OS X and Linux.

  • pv Y2013M11D06.sql | wc
  • 2.47GiB 0:00:11 [ 212MiB/s] [========================================================================>] 100%
  • 98159 122143517 2648450403

It took about 11 seconds to complete, and as it completed the progress bar made its way across the window.

Note that I don’t have MacPorts or HomeBrew installed; a simple ./configure and make worked fine.

File this under: Why did I not already know about this!

Nisus HTML script now handles floating content—Saturday, September 28th, 2013

My Nisus simple HTML publish script now handles floating images and floating text boxes. This means that floating images are no longer lost, and neither are tables or other text inside of floating text boxes.

This was made possible by several new features in Nisus’s macro language. For images, the script no longer has to extract the image from RTF. Selections have an enclosedInlineImages and an enclosedFloatingContents method, making it simple enough to grab any images in a selection:

[toggle code]

  • $currentSelection = $thisDocument.textSelection
  • #are there any images?
  • $images = $currentSelection.enclosedInlineImages
  • If $images
    • ForEach $image in $images
      • SaveImage($image)
    • End
  • End
  • $floats = $currentSelection.enclosedFloatingContents
  • If $floats
    • ForEach $float in $floats
      • If $float.isImage
        • SaveImage($float.image)
      • End
    • End
  • End

And images themselves are now objects with properties and methods, including the ability to write themselves out to a file:

[toggle code]

  • Define Command SaveImage($image, $imageFolder)
    • File.createFolderWithPath $imageFolder
    • $imagePath = $image.writeWebImageToFolderAtPath $imageFolder
    • $imageFileName = $imagePath.lastFilePathComponent
    • $imageName = $imageFileName
    • Begin Perl
      • use URI::Escape;
      • $imageFileName = uri_escape($imageFileName);
    • End
    • $imageHTML = "<img src=\"images/$imageFileName\" alt=\"$imageName\" />"
    • return $imageHTML
  • End

The writeWebImageToFolderAtPath ensures that the image is in a standard web format, regardless of its format in the document. This means that embedded PDFs and EPS images will be written to a more standard format, currently JPEG.

The ability to define commands contributed heavily to the main improvement of this version. It allowed me to separate out the code that handles text conversions, which in turn allowed me to call it both for text in the normal document flow and on text in floating text boxes.

AppleScript as shell scripting language—Tuesday, September 10th, 2013

I just put two very obvious things together that I always knew, but hadn’t thought about at the same time. Since /usr/bin/osascript can send arguments to AppleScript text scripts, and since the shell automatically runs scripts through the first comment-bang if it exists as an executable, AppleScript can be used from the command line without even needing things like the long-dormant appscript.

[toggle code]

  • #!/usr/bin/osascript
  • on run names
    • repeat with name in names
      • log "Hey, " & name
    • end repeat
  • end run

Then, run it with some arguments, and:

  • $ bin/hey Fred Wilma Barney
  • Hey, Fred
  • Hey, Wilma
  • Hey, Barney

This even runs without a logged-in user. Now, obviously, with the strength of AppleScript being contacting running applications, using this without a logged-in session isn’t going to be a common solution.

Here, for example, is a script that uses Nisus Writer Pro to educate text in a text file, using Nisus Writer’s Plain to Smart quotes menu item:

[toggle code]

  • #!/usr/bin/osascript
  • -- educate text using Nisus
  • on run arguments
    • set workingDirectory to do shell script "pwd"
    • set results to ""
    • repeat with flatfile in arguments
      • set currentFile to workingDirectory & "/" & flatfile
      • set currentFile to POSIX file currentFile
      • log currentFile
      • set currentText to theContents for currentFile
      • tell application "Nisus Writer Pro"
        • set workspace to make new document
        • set text of workspace to currentText
        • --if Nisus had to start up, we need to activate it to get rid of the About window
        • if name of window 1 is "About Nisus Writer Pro" then
          • activate
        • end if
        • Do Menu Macro with macro "Edit:Select:Select All"
        • Do Menu Macro with macro "Edit:Convert:Plain Quotes to Smart Quotes"
        • set educatedText to text of workspace as text
        • set results to results & "\n" & educatedText
        • --display dialog educatedText
        • close workspace saving no
      • end tell
    • end repeat
    • copy results to stdout
  • end run
  • --read a file and return its contents

A couple of tricks and notes:

  • It uses the “log” command to log to standard error.
  • It uses “copy variable to stdout” to write to standard output. However, in AppleScript, stdout is some sort of pseudo variable; it only holds the last “copy to” and it cannot be appended to because it isn’t defined. So I’m saving the output text in a variable and then copying that variable at the end of the script.1
  • There might be a better way to get the current working directory inside of AppleScript, but I don’t know what it is. It feels kind of like cheating to get it by using “do shell script”. This is a shell script.
  • The “Do Menu Macro” commands don’t target a specific window. They’ll take the top one, which should be the one we just created. Currently, Nisus must have been activated or visited at least once for Select All to work. Otherwise, it sticks on the “About” window that applications often display on startup. This script only activates Nisus if the About window is the first window; this means that it only pulls out of Terminal if Nisus had to start up. Otherwise, there is no need for a focus shift.
  • I’m assuming that all files I’m looking at will be in UTF-8, so I’m specifying that when I read it in. If you leave out the “as «class utf8»” then AppleScript assumes something else, I think MacRoman.
  • Display Dialog works, too. I used it here for testing whether I was successfully getting the educated text back from Nisus. It only works if it’s in another application’s context, however, such as here where it is in Nisus’s context.
No more Twitter on masthead—Wednesday, July 10th, 2013

I’ve removed my twitter feed from the masthead. When Twitter offered an RSS feed, putting up the latest two tweets was trivial. I’ve looked at their new rules for using the JSON feed, and don’t feel any desire to make sense of them.

I wouldn’t even have bothered mentioning it, but in the process of trying to find out what was possibly going through their heads removing something so simple and easy to use, I ran across some very good blog entries talking about the wonders of RSS.

From Battle for the planet of the APIs:

In the web’s early days, AOL offered an alternative. “You don’t need that wild, chaotic lawless web”, it proclaimed. “We’ve got everything you need right here within our walled garden.”

Of course it didn’t work out for AOL. That proposition just didn’t scale, just like Yahoo’s initial model of maintaining a directory of websites just didn’t scale. The web grew so fast (and was so damn interesting) that no single company could possibly hope to compete with it. So companies stopped trying to compete with it. Instead they, quite rightly, saw themselves as being part of the web. That meant that they didn’t try to do everything. Instead, you built a service that did one thing really well—sharing photos, managing links, blogging—and if you needed to provide your users with some extra functionality, you used the best service available for that, usually through someone else’s API… just as you provided your API to them.

Then Facebook began to grow and grow. I remember the first time someone was showing me Facebook—it was Tantek of all people—I remember asking “But what is it for?” After all, Flickr was for photos, Delicious was for links, Dopplr was for travel. Facebook was for… everything… and nothing.

I just didn’t get it. It seemed crazy that a social network could grow so big just by offering… well, a big social network.

But it did grow. And grow. And grow. And suddenly the AOL business model didn’t seem so crazy anymore. It seemed ahead of its time.

From Lockdown:

This is how RSS and Atom have always worked: you put in some effort up front to get the system built,2 and in most instances, you never need to touch it. It just hums along, immune to redesigns, changing APIs, web-development trends, and slash-and-burn executives on “sunsetting” sprees.

RSS grew up in a boom time for consumer web services and truly open APIs, but it especially spread like wildfire in the blogging world. Personal blogs and RSS represented true vendor independence: you could host your site anywhere, with any software. You could change those whenever anything started to suck, because there were many similar choices and your readers could always find your site at the domain name you owned.

And from The web we lost:

MathPad is back!—Thursday, February 28th, 2013

Just an FYI, MathPad is back! Version 3.0.4 supports Mac OS X 10.6 and up, and works on Intel Macs without Rosetta. You’ll want to read the What’s New file; for example, I’ve been using .mathpad as the extension to get Leopard/Snow Leopard to associate my MathPad files with MathPad. In 3.0.4, the extension is officially .mp.

Another major change is that the default file format is RTF instead of text. You can save as text but there is unfortunately no preferences setting to prefer text over RTF. MathPad continues to read text files fine.

Some neat changes (I think) include the ability to use π instead of ‘pi’ in calculations.

There are several other changes as well, likely related to the switch from Carbon to Cocoa.

Custom ModelForms with django-ajax-selects—Friday, February 1st, 2013

I have a Model for URLs that contains:

[toggle code]

  • class Destination(models.Model):
    • path = models.CharField(max_length=140)
    • host = models.ForeignKey(Host, default=1)
    • query = models.CharField(max_length=240, blank=True)
    • secure = models.BooleanField(default=False)
    • class Meta:
      • ordering = ['host__name', 'path']
      • unique_together = (('path', 'host', 'secure', 'query'),)
    • def save(self, *args, **kwargs):
      • if self.path.startswith('/'):
        • self.path = self.path.lstrip('/')
      • return super(Destination, self).save(*args, **kwargs)

The problem is when the stripping results in an existing URL. Model saves, whether in models.Model or admin.ModelAdmin, happen after validation, so instead of a nice “this URL already exists” I get a server error. Django thinks this isn’t a duplicate because it already passed its validation, and then I went ahead and changed it.

This is easily fixed in the Admin model, by using a custom ModelForm:

[toggle code]

  • class DestinationForm(forms.ModelForm):
    • class Meta:
      • model = Destination
    • def clean_path(self):
      • path = self.cleaned_data['path']
      • if path.startswith('/'):
        • path = path.lstrip('/')
      • return path
  • class DestinationAdmin(admin.ModelAdmin):
    • form = DestinationForm

Now, adding a new Destination results in the correct “Destination with this Path, Host, Secure and Query already exists.”

But it still fails in the “+add” pop-up form. I’m using django-ajax-selects, and the error is in the views.py for django-ajax-selects:

Older posts.