I borrowed a serial UPC scanner from work today for some Friday night hacking. It’s a ReaderWare LaserChamp/Flic Barcode reader re-branded under the SerialIO name. It came with a standard serial cable and didn’t have a manual.

First, I booted to my Ubuntu partition to see the raw data from the scanner.

$ cat /dev/ttyS0

No dice. I did some searching and found that the scanner likely ran at 4800 baud.

$ setserial /dev/ttyS0 baud_base 4800
$ cat /dev/ttyS0

^B043396143234

Ah, some data. I removed the control characters and had myself a UPC symbol. I wrote a script and scanned a bunch of books, CDs, and other crap sitting on my desk. I also found a manual for the scanner, which had some settings symbols¹.

I booted to my XP partition and started Cygwin. I wrote a Perl script to mimic what I did on Linux, except using the Win32::Serial module. I fired up CPAN to grab the library, but CPAN was down. I switched to ActiveState’s Perl distribution and grabbed their Win32:: PPM. I finished the script and tested it. It worked.

A while later I checked CPAN again, grabbing the Win32:: module using Cygwin’s Perl. It didn’t build. This seems typical for Cygwin and CPAN/Perl modules². I moved on.

I switched to C# to see how it compared to Perl and the Linux/Bash hack. I whipped together a simple application³ that listened for data on a serial port, cleaned it up, and copied it to the clipboard. It worked⁴.

I also attempted to hook the C# code up to the Amazon API (to get ASINs), but found the networking libraries more complicated than I had time for⁵.

My favorite approach? Hands down, I liked the Bash script⁶ best. It was one line of code, including cleaning it up with sed. It’s not something I can ship, but it’s the way I’ll scan all of my books/CDs/DVDs.

The Perl Win32::Serial hack was longer (a dozen lines), but it also worked. It was no better than the 1-line Bash script, so I tossed it.

The C# hack consisted of a single form and its supporting class (a few dozen lines of code). It worked, and is easy to distribute. I’ll use this if I’m stuck doing something in Windows.

Total time for all 3 hacks? 3 hours and 25 minutes⁷.

1. You print the page and scan the symbols to change the scanner’s settings ↩
2. Most of the tests failed for the Win32::Serial module ↩
3. And the .NET build of it ↩
4. C#’s serial code was nicer than Perl’s, but not as nice as the devfs hack ↩
5. Perl’s networking is much nicer than C#’s ↩
6. Bash and devfs’s /dev/ttyS0 ↩
7. My wife and kids were out for the evening ↩

I’ve heard a few people wonder how Apple ported Webkit to Windows, so I cracked open depends.exe and took a look at what Safari was linking to. A few of the more interesting dependencies:

• Webkit (via CoreGraphics)
• CoreFoundation (and CF-Net)
• PThreads
• MSVCP8 (interesting to see them use MSVC SP2)
• SQLite3

So it’s not based on QT, and it doesn’t seem to match the iTunes set (which has no Core* dependencies). Looks a lot like a straight port of some of the base Cocoa libraries.

Here’s an idea that I haven’t had a chance to hack at yet: Php++. Take the base language, clean it up using a simple pre-processor, and spit out regular Php. Think of it like Rails, but inverted. Instead of creating a few mini-languages to replace HTML/CSS/Javascript/SQL generation, find a way to string these languages together in a cleaner way. An exoskeleton versus an endoskeleton.

I don’t have anything against Ruby (it’s a cleaner, crisper Perl), and it’s not like I don’t appreciate what Rails tries to do. What gets in my way is that Rails introduces new domain-specific-languages that aren’t as good as what they replace.

What Rail’s DSLs intend to do is great: automating database migrations, schema/code synchronization, managing project skeletons, Javascript integration, and more. The problem is that these DSLs are specific to Rails, and represent non-portable knowledge. You have to relearn how to do JOINs, UNIONs, and POSTS. Not that the Rails way isn’t clever, but it’s a layer between a set of languages that are already really good. My question is, what’s wrong with writing SQL, HTML, and JS? I think the answer is that their intersection is messy, and you duplicate your application’s domain knowledge between each of the specific languages.

So Rails helps to enforce the One Definition Rule, but it does so by inventing new mini-languages. Rails does it well, which is forgivable, but I’m thinking that the same end is possible while still embracing SQL, HTML, and Javascript. Anything you know about these languages should still apply, so anything you learn using the tool would be useful elsewhere.

Enter Php++

Add a small number of keywords similar to the QT C++ extensions that make it easy to generate sensible code from your SQL/HTML/JS. For example:

class student {
    public get address() = "SELECT * FROM Students WHERE id = '$sid'";
    public set address($street, $number, $etc) = "INSERT ...";

    // The display portions rely on get/set
    public edit show_edit_form() default(); 
    public edit show_inline_edit() default(ajax);
    public view display_full_address() default(); 

}

The extensions would also take care of the mess around escaping SQL, and could switch between various SQL backends without much trouble (mysql_*, Pear::DB, etc.). And the Php++ processor would do the right thing, and would generate real, readable Php.

Now my only problem is finding the time to hack at it.

Let’s face it, the default Php MySQL interfaces are pretty weak. A few of the shortcomings:

• No prepared statements
• In lacking prepared statements, requires the use of escaping functions for any SQL containing POST/GET variables
• Read functions overwrite duplicate column names in returned data when reading rows from joined tables
• Has a very verbose API, with many long parameter lists
• Is not class based, so everyone bakes their own abstraction

Compared to the Perl DBI, for example, Php’s mysql_ functions are absolutely horrid. Luckily there are many great alternatives, as I discovered this weekend with a bit of research.

The PDO

Php’s PDO functions are similar to Perl’s DBI, sporting a reasonable API and supporting a number of databases.

Php’s improved MySQL functions

Php has an improved set of MySQL functions (the mysqli_ functions) that add both a simple class wrapper and a parallel, improved API. The new API supports prepared statements, and has a cleaner interface layout. The new APIs still munge the fetch_assoc functions (they still overwrite), but are otherwise a great improvement.

The improved MySQL functions are only available in Php 5, and I’ve read warnings that it’s a pain to run both the msql_ and mysqli_ functions from one Php installation.

Pear’s MDB2

The MDB2 package from the Pear repository replaces their previous DB package, and provides a similar interface to Perl’s DBI. I find the interface a bit bigger than my needs present, but it is a very complete and mature abstraction.

In the real world

All of these libraries are available on most platforms and from most reasonable webhosts (like Dreamhost).

My first choice so far is the improved mysqli_ class/functions, then the built-in PDO library. I’ll spend a few weeks with them and then post a follow-up.

I was showing a friend a line of code I wrote tonight. It’s nothing special, just a progress indicator that is displayed when a form is posted. I realized, though, that the one line of code touched four different ‘languages’, and a larger number of areas of know-how:

$('view').innerHTML = '<img src="/images/indicator.white.gif" />';

1. Javascript.
2. HTML (markup language)
3. CSS (formatting language)
4. PHP

The line is emitted Javascript (reacting to a DOM event in an HTML form), produced by Php. The Javascript is writing some HTML to a DOM node, which is styled based on a set of defined CSS classes.

The line implies knowledge of the W3 DOM, and requires understanding the scriptaculous and prototype library (which hides the wonders of the DOM access and XmlHTTPRequest). Putting together the application quickly also required some understanding of Unix, which itself touches several tools. If the form were doing anything interesting, it would also bang up against SQL. And you know what? It’s not a half-bad way of building an application with a user interface.

I’ve been hacking at another incarnation of my natural-language markup plugin (smarky), and decided to put up a quick web-based test interface. The web interface took less than 10 minutes to whip together, containing only a handful of code and a small bucket of markup.

I found an interesting Php behaviour tonight. The language allows you to call a member function as if it were a static function, something you can accidentally hide when calling callback functions:

// Oops, this calls a non-static member statically as sm::blocks
preg_replace_callback(
       "/(?<=\n\n|\r\n\r\n|\r\r|\A)(.*?)(?:\n\n|\r\n\r\n|\r\r|\Z)/sm",
       array("sm", "blocks"),
       $text);
}

What I really meant was:

array($this, "blocks")

When you call a static function in a Php class, it has the side-effect of wiping out the class context. Without the class context, you can’t make other calls to class members. In my case above, Php allowed me to break the class context (calling a non-static as static) without warning, which caused a perfectly legitimate class member function fail:

function text($m) {
    $this->inlines($m);
}

Which fails with: Using $this when not in object context.

Instead, Php should warn: Calling a non-static member as static. This might not be what you meant to do.

There are some weeks that I try to cram too much into my head. I can always tell it’s getting full too, when it begins to refuse new material and I start to get cranky. Today I’m getting cranky.

This week I looked at a bunch of new technologies, I made significant progress on two separate projects at work, and I started work on a few personal projects that have been waning. I also started reading a textbook that arrived in the mail on Friday, Software Conflict 2.0, which resulted in a great deal of well-needed reflection.

The technologies I crammed in my head all proved to be deceptive, as they often are. Each promised to revolutionize how something is done, without any of the pain of the other approaches. Each inevitably made some small aspect of a problem trivial, at the cost of other things. The search was fruitful, though I’m not sure my muse has finished digesting it all.

The theme of this week’s brain-freeze: While a framework may make a few things easier, it always makes at least a few things harder.

I was going to continue to work some more tonight, but instead I think I’ll give my head a break. My muse has some catching up to do.

I’m forgetting my Perl already.

I’ve been burried in C++ and SQL land for a few months now, and today one of our younger developers asked me a simple Perl/Regex question. I was stumped. I misplaced my memories on the differences between regex in Perl, Php, Grep, and others (they were somehow all jumbly in my head). I happened upon an answer; it stumbled out of my mind, and only by chance was it correct. I realized that I was forgetting Perl, one of my favorite languages.

Luckily forgetting something you know isn’t at all like not knowing something you should, and with some practice my skill popped back to the surface.

Tonight I whipped up a few scripts to extract several dozen constants from some C++ sources, to generate some SQL statements. At first my Perl memories trickled back into view (I confused a few small things like C++’s continue for Perl’s next), but eventually it all came back and I was able to hack several dozen lines of useful script.

I have to remember to take time to practice with the tools I of the trade, otherwise they may start to disappear from my limiting human consciousness.

It’s one of those regex-laden days, and I’m really starting to grok more complicated expressions:

^(?:\s+|)<(\w+)(?:\s+|)((?:.*?|))>(.*?)(?:<\/(.*?)>|)(?:\s+|)$

This expression parses a line that contains HTML tags based on the following logic, expecting that:

1. There will be a start tag near the beginning of the line, possibly padded on the left with spaces that are ignored
2. The opening tag may contain some HTML parameters
3. There may be a closing tag on the line
4. There may be spaces on the right of the closing tag that will be ignored

The expression will parse the following example into 4 parts:

<h1 id="test">This is a test</h1>

1. h1
2. id=”test”
3. This is a test
4. h1

Learning regex to the point of being able to write complex expressions has taken a couple of years, but has been well worth the effort. To define the same parsing logic in C or C++ (using standard mechanisms) would take 20-30 minutes, and would occupy a page of code. You just have to remember that a regex is a small script, and that it should be tested (and documented) like one.

Regex is like a lot of little languages too (like SQL, bash, m4). It’s terribly useful, succinct, and worth having in your toolkit. It’s not something to hide in layers of abstraction either, rather it’s something that deserves use alongside your ‘real’ tools. I find that developers are in the habit of hiding (or hiding from) little languages, something that results in the too-many-elbows syndrome: insulating yourself from the real power of your tools, making things more complicated in the process.

Simple, in the end, is in the knowledge of the beholder. If you understand regex, code that contains it can be simpler.

I always forget a few things about Perl regular expressions. Regexes are a large specification, I only ever use a portion of it at any one time, and I’m forgetful (there’s just too much to remember all at once).

Luckily, unix tools allow me to easily find nuggets of previous knowledge. Today’s exapmle:

find -name "*pl" -exec grep =~ {} \;
          1              2  3  4

1. Look through all my old Perl scripts
2. Search in the scripts
3. Look for lines with regular expressions
4. In the files we found

In the five seconds it takes me to type the find command , I have a full list of thousands of regular expressions. I can forget most of what I know, as long as what I’ve done is stored in text on a system with good text tools.