Below are the 20 most recent journal entries recorded in Jp Calderone's LiveJournal:

[ << Previous 20 ]

Thursday, June 4th, 2009
9:48 am	April - May Reading List Green Mars. Kim Stanley Robinson. Blue Mars. Kim Stanley Robinson. (Comment on this)
Tuesday, May 26th, 2009
6:32 pm	Note To Self: co_filename vs __file__ Python's traceback formatting code uses the co_filename attribute of code objects. However tempting the __file__ attribute may seem, don't use it if you want to get results that agree with the built-in traceback formatting code. (Comment on this)
Thursday, April 2nd, 2009
10:12 am	January through March Reading List House of Leaves. Mark Danielewski. Jhegaala. Steven Brust. Red Mars. Kim Stanley Robinson. (Comment on this)
Wednesday, February 11th, 2009
5:12 pm	Three Software Failures I ran out of space on / on my desktop today. As a result of this three obviously wrong things happened. First, joe truncated a file to zero bytes and then segfaulted. Second, emacs truncated a file I was working on to zero bytes. Third, sticky notes lost all of my notes. If you write and release software that behaves like this, you need to be embarrassed. (2 Comments |Comment on this)
Wednesday, January 7th, 2009
9:58 pm	September through December reading list Dogs of God: Columbus, the Inquisition, and the Defeat of the Moors. James Reston Jr. Zero: The Biography of a Dangerous Idea. Charles Seife. The Best Of H.P. Lovecraft. H.P. Lovecraft. The Given Day. Dennis Lehane. The Sun Kings. Stuart Clark. 1453: The Holy War for Constantinople and the Clash of Islam and the West. Roger Crowley. (Comment on this)
Wednesday, December 10th, 2008
12:09 pm	What is the SigQ field of /proc/PID/status in Linux? I had to read kernel source to figure this out, and when I did I learned the value isn't at all what I thought, so I thought I'd let everyone else know. From fs/proc/array.c: qsize = atomic_read(&p->user->sigpending); ... buffer += sprintf(buffer, "SigQ:\t%lu/%lu\n", qsize, qlim); where p is a struct task_struct p. From include/linux/sched.h: struct task_struct { ... /* process credentials */ ... struct user_struct *user; and /* * Some day this will be a full-fledged user tracking system.. */ struct user_struct { ... atomic_t sigpending; /* How many pending signals does this user have? */ So the SigQ value in each process's status file is the total number of signals pending delivery across all processes owned by the UID of the process (or maybe the UID which started the process, I don't know how the user field of task_structs interacts with setuid). Pretty far from what I've been assuming it meant, the number of signals pending delivery for each individual process. :/ Current Mood: indifferent (Comment on this)
Tuesday, December 9th, 2008
12:44 pm	ioctl TIOCSTTY EIO What does it mean if a TIOCSTTY ioctl fails with EIO (errno 5) on Linux? Current Mood: enraged (9 Comments |Comment on this)
Sunday, September 14th, 2008
8:12 pm	August reading list Singularity Sky. Charles Stross. Anvil of Stars. Greg Bear. Dune. Frank Herbert. (1 Comment |Comment on this)
Wednesday, August 6th, 2008
7:19 pm	Python and XPath Quite some while ago, I decided to experiment with writing unit tests for some Nevow-based application code using XPath instead of DOM. DOM-based assertions in unit tests tend to be quite verbose compared to the XPath equivalent. Beyond that, though, the XPath seems to be able to express the intent of the test more accurately. It doesn't rely on as many irrelevant details (like how many TextElements some CDATA is broken up into) - or at least that's the idea I wanted to try out. Unfortunately I only wrote a few tests like this and shortly afterwards I started focusing on non-XML tasks. As far as the experiment went, I think it went well, but it's basically incomplete at this time. That's not what this post is about. A while ago, Ubuntu's Hardy Heron was released. In Hardy, the Python package xml.xpath has vanished. Apparently the package was unmaintained for some time and either Ubuntu or Debian or some combination of the two decided that it would no longer be supported. Sad for me, because now all my XPath-based tests are broken on Ubuntu, which happens to be my primary platform. It seems there are some other XPath libraries for Python available in Hardy, so today I tried to update my tests so that they will work on systems which still have xml.xpath but can also try to use a newer XPath library if xml.xpath is unavailable. Unfortunately (again) this turned out to be more difficult than I expected. The fall-back library I selected is lxml.etree. The first problem I encountered is that the Python API for processing XPath in lxml.etree is incompatible with the API in xml.xpath. So instead of just changing a couple imports as I had hoped I would be able to do, I wrote a couple thin wrappers to expose both APIs in the same way. The second problem I encountered is that the result objects returned by lxml.etree have a different API than the result objects returned by xml.xpath. So I changed my thin wrappers to be more specific, extracting the exact data my unit tests required, rather than being somewhat more general and letting each test grab what it needed. This meant widening the API to two wrappers from one. try: from xml.xpath import Evaluate except ImportError: from lxml.etree import XPath, fromstring def evaluateXPath(path, document): return XPath(path).evaluate(fromstring(document)) def evaluateTextXPath(path, document): return evaluateXPath(path, document)[0] else: def evaluateXPath(path, document): return Evaluate(path, minidom.parseString(document)) def evaluateTextXPath(path, document): return evaluateXPath(path, document)[0].wholeText After solving these two problems, I had two unit tests passing on my Hardy system. Not a complete failure. I have more than two unit tests that use XPath, though. As it turns out, the next set of tests I looked at need to examine the values of attributes of nodes. This is another area where xml.xpath and lxml.etree present different APIs. If I want to make these tests pass, I need to write another wrapper function to expose attributes in a uniform manner. Here's where I give up on fixing this for the moment. It seems it's not reasonable to expect to transparently switch between xml.xpath and lxml.etree. Perhaps there's another XPath library out there I can use instead of lxml.etree. Anyone have any tips they'd like to share? Current Music: The Lovin' Spoonful - Bes' Friends (8 Comments |Comment on this)
Sunday, July 27th, 2008
5:17 pm	May through July Reading List Against The Day. Thomas Pynchon. Nature Girl. Carl Hiaasen. HaltinG StatE. Charles Stross. (Comment on this)
Tuesday, July 1st, 2008
10:03 am	TCP in the browser? Allen brought to my attention a blog post by Michael Carter about how Orbited now offers a JavaScript API for making TCP connections. I'm not sure how useful this is right now (and I actually mean that I'm not sure, not that I think it's useless and am trying to avoid saying so), though I can see how if, at some future date it's possible to make TCP connections without going through a proxy, apps written this way will suddenly make a lot of sense and be easy to port from apps already based on an API like this. Anyhow, I thought this would be simple to implement with Athena, but I wanted to see how simple. Five minutes later I had this untested code: # athenatcp.py from twisted.internet.protocol import ClientCreator, Protocol from twisted.internet import reactor from nevow.athena import LiveElement, expose class TCPElement(LiveElement): jsClass = 'Network.TCP' @expose def connect(self, host, port): def connected(proto): self.proto = proto self.proto.dataReceived = self.dataReceived self.proto.connectionLost = self.connectionLost cc = ClientCreator(reactor, Protocol) d = cc.connectTCP(host, port) d.addCallback(connected) return d @expose def write(self, bytes): self.proto.transport.write(bytes) def dataReceived(self, data): self.callRemote('dataReceived', data) def connectionLost(self, reason): self.callRemote('connectionLost', reason.getErrorMessage()) // athenatcp.js // import Nevow.Athena Network.TCP = Nevow.Athena.Widget.subclass('Network.TCP'); Network.TCP.methods( function connect(self, host, port) { return self.callRemote('connect', host, port); }, function dataReceived(self, bytes) { // override this }, function connectionLost(self, reason) { // override this }); It'd probably be better if connect were actually a factory function returning TCP instances, but the general idea would be the same. Anyone think this is a really great idea? Anyone want to turn this spike into a real library for Athena? (5 Comments |Comment on this)
Saturday, June 28th, 2008
6:39 pm	re: inventory meme Glyph tagged me in his Inventory meme several weeks back. At first, I didn't respond because I was too busy. Then, because I forgot. Then, because I decided it would be cheating to wait until I had the maximum number of items possible and then post. Just now I happen to have been reminded of the meme, and I think I can spare the three minutes total this post is going to take, and I'm totally unprepared. So, here it is. Your hands are empty. You are completely naked. (1 Comment |Comment on this)
Saturday, April 26th, 2008
1:11 pm	February through April Reading List Something Wicked This Way Comes.  Ray Bradbury. The Eyre Affair.  Jasper Fforde. The Star Rover.  Jack London. Travels With Herodotus.  Ryszard Kapuściński. Omnivore's Dilemma.  Michael Pollan. Lankhmar.  Fritz Leiber. Suite Française.  Irène Némirovsky. Persepolis: The Story of a Childhood.  Marjane Satrapi. Persepolis 2: The Story of a Return.  Marjane Satrapi. Maus 1: My Father Bleeds History.  Art Spiegelman. Maus 2: And Here My Troubles Began.  Art Spiegelman. Current Music: The Velvet Underground - Some Kinda Love (Comment on this)
Friday, April 11th, 2008
12:21 pm	pyOpenSSL 0.7 final released Alright, it's done. Here are the highlights since 0.6: Bug fixes: Memory leak in X509.get_pubkey eliminated Memory leaks in X509Name.__getattr__ and X509Name.__setattr__ eliminated RuntimeWarning from X509Name comparison eliminated X509Name reference counting issues resulting in memory corruption eliminated Uninitialized PKeys are now rejected by X509Req signature APIs Memory leaks in X509Name.__getattr__ and X509Name.__setattr__ eliminated New features: SSL_get_shutdown and SSL_set_shutdown exposed as Connection.get_shutdown and Connection.set_shutdown SSL_SENT_SHUTDOWN and SSL_RECEIVED_SHUTDOWN exposed as SSL.SENT_SHUTDOWN and SSL.RECEIVED_SHUTDOWN X509_verify_cert_error_string exposed as OpenSSL.crypto.X509_verify_cert_error_string X509.get_serial_number and X509.set_serial_number now accept long integers Expose notBefore and notAfter on X509 certificates for inspection and mutation Expose low-level X509Name state with X509Name.get_components Expose hashing and DER access on X509Names Current Music: Kings Of Leon - Velvet Snow (Comment on this)
Saturday, March 22nd, 2008
2:13 pm	pyOpenSSL 0.7a1 Hot off the presses, pyOpenSSL 0.7a1 released. Give it a spin and let me know how it goes. Current Music: Grateful Dead - Not Fade Away (Comment on this)
Wednesday, February 6th, 2008
2:07 pm	Ever relevant It is good to re-read Alan Perlis' epigrams in programming from time to time. Some which stand out to me today: 32. Programmers are not to be measured by their ingenuity and their logic but by the completeness of their case analysis. 41. Some programming languages manage to absorb change, but withstand progress. 64. Often it is the means that justify the ends: Goals advance technique and technique survives even when goal structures crumble. 94. Interfaces keep things tidy, but don't accelerate growth: Functions do. One which I don't think has ever stood out to me before: 20. Wherever there is modularity there is the potential for misunderstanding: Hiding information implies a need to check communication. And of course, there are the old favorites: 11. If you have a procedure with ten parameters, you probably missed some. 95. Don't have good ideas if you aren't willing to be responsible for them. 101. Dealing with failure is easy: Work hard to improve. Success is also easy to handle: You've solved the wrong problem. Work hard to improve. 117. It goes against the grain of modern education to teach children to program. What fun is there in making plans, acquiring discipline in organizing thoughts, devoting attention to detail and learning to be self-critical? (1 Comment |Comment on this)
Saturday, February 2nd, 2008
8:14 pm	October through January Reading List Smoke and Mirrors. Neil Gaiman. The Long Tomorrow. Leigh Brackett. Norstrillia. Cordwainer Smith. Nova. Samuel R. Delany. Iron Sunrise. Charles Stross. The Lies of Locke Lamora. Scott Lynch. The Road. Cormac McCarthy. Paradise Lost. John Milton. Lord of Light. Roger Zelazny. (Comment on this)
Friday, January 4th, 2008
8:48 am	notes on netgear support website comments are mangled incorrectly, doubling single quotes, presumably in a confused attempt to prevent SQL injection attacks. page caching behavior is wrong, so after a ticket is updated by netgear support staff, the update will not appear unless the browser is made to disregard its cache. timestamps on comments are wrong, perhaps by roughly (UTC - PST) hours. netgear support staff can't actually fix any problem you have. Current Mood: pissed off Current Music: Fallible, Blues Traveler (Comment on this)
Thursday, January 3rd, 2008
9:18 pm	One in twenty eight and falling... Asteroid on near-miss course with Mars No one taught these guys to label their plots, unfortunately (nice of them to put "mars" and "sun" on there though). To make it clearer, the cyan line is the probable trajectory of 2007 WD5. White dots indicate a possible position at the time it would intercept mars' orbital, if it does. Will there be a collision? Probably not. One in twenty eight odds, though? Those are most certainly not astronomical odds. (Comment on this)
Friday, December 21st, 2007
4:58 pm	Filesystem structure of a Python project Do: name the directory something related to your project. For example, if your project is named "Twisted", name the top-level directory for its source files Twisted. When you do releases, you should include a version number suffix: Twisted-2.5. create a directory Twisted/bin and put your executables there, if you have any. Don't give them a .py extension, even if they are Python source files. Don't put any code in them except an import of and call to a main function defined somewhere else in your projects. If your project is expressable as a single Python source file, then put it into the directory and name it something related to your project. For example, Twisted/twisted.py. If you need multiple source files, create a package instead (Twisted/twisted/, with an empty Twisted/twisted/__init__.py) and place your source files in it. For example, Twisted/twisted/internet.py. put your unit tests in a sub-package of your package (note - this means that the single Python source file option above was a trick - you always need at least one other file for your unit tests). For example, Twisted/twisted/test/. Of course, make it a package with Twisted/twisted/test/__init__.py. Place tests in files like Twisted/twisted/test/test_internet.py. add Twisted/README and Twisted/setup.py to explain and install your software, respectively, if you're feeling nice. Don't: put your source in a directory called src or lib. This makes it hard to run without installing. put your tests outside of your Python package. This makes it hard to run the tests against an installed version. create a package that only has a __init__.py and then put all your code into __init__.py. Just make a module instead of a package, it's simpler. try to come up with magical hacks to make Python able to import your module or package without having the user add the directory containing it to their import path (either via PYTHONPATH or some other mechanism). You will not correctly handle all cases and users will get angry at you when your software doesn't work in their environment. (6 Comments |Comment on this)