Mimsy Were the Borogoves

Hacks: Articles about programming in Python, Perl, PHP, and whatever else I happen to feel like hacking at.

Django QuerySet Heisenberg gotcha

Jerry Stratton, April 23, 2009

We have a task management system in Django that I was modifying to email assignees when they are added to or removed from a task. The changes worked fine when testing on the development server, but refused to notice any changes to the assignee list in the live server.

See if you can tell why:

[toggle code]

  • 1. class projectBaseForm(forms.ModelForm):
    • 2. def save(self, manager=None):
      • 3. if self.instance.id:
        • 4. previousAssignees = self.instance.assignees.all()
      • 5. else:
        • 6. previousAssignees = []
      • 7. #print "Previous assignees:", previousAssignees
      • 8. newVersion = super(projectBaseForm, self).save()
      • 9. self.assigneeNotifications(previousAssignees, newVersion)
      • 10. return newVersion
    • 11. def assigneeNotifications(self, previousAssignees, task):
      • 12. assignees = task.assignees.all()
      • 13. #construct new assignee list
      • 14. newAssignees = []
      • 15. for assignee in assignees:
        • 16. if assignee not in previousAssignees:
          • 17. newAssignees.append(assignee)
      • 18. #construct removed assignee list
      • 19. removedAssignees = []
      • 20. for assignee in previousAssignees:
        • 21. if assignee not in assignees:
          • 22. removedAssignees.append(assignee)
      • 23. if newAssignees:
        • 24. task.taskMail(newAssignees, mailTemplate="projects/mails/assigned.mail")
      • 25. if removedAssignees:
        • 26. task.taskMail(removedAssignees, mailTemplate="projects/mails/deassigned.mail")

If the assignees before making a change are George, Bill, and Ted, and the assignees after making the change are George, Bob, and Jay, what are the values of previousAssignees and assignees in the two for loops in the assigneeNotifications method at lines 15 and 20?

The answer depends on whether or not the “print "Previous assignees:", previousAssignees” line is commented or uncommented. One of the key features of Django’s QuerySets is that they aren’t evaluated until they “need” to be evaluated. But that also lends a somewhat non-sequential Heisenbergian nature to the code. In the version as it’s listed above, previousAssignees isn’t evaluated until it is used in the assigneeNotifications method. By this time, the previous assignees have been lost—the save has already happened putting the new assignees in place!

Observing a system changes the system. During the initial testing, I used that print line to see what was happening. And the act of printing previousAssignees caused the QuerySet to evaluate, producing the result I wanted. When I moved it to the production server, I commented that line out. The fix is simple:

[toggle code]

    • def save(self, manager=None):
      • if self.instance.id:
        • #use list to force it to perform the query immediately
        • #otherwise, it waits until after the changes have been made to perform the query
        • previousAssignees = list(self.instance.assignees.all())
      • else:

As described on the Django QuerySet API page, I need to force the query to evaluate if I want it to evaluate immediately, using something like the list() method.

  1. <- Django code tag
  2. Mailman distutils ->