yeago works

I recently moved a decently sized database (400,000 records, 40 tables) from MSSQL to a Debian server running MySQL. I’m pretty meticulous when it comes to saving old data, so I wanted to keep the integrity exactly right. In general, I exported my old data and loaded it into a temporary project and used django to create models on my old data. Then I wrote a python script using Django’s DB library to loop through this old data and get it into my new format. Super easy!

1) Exporting the data from the old database

I happened to use the MySQL Migration Toolkit, a rather wonderful tool. It wasn’t too difficult to use–I ended up with a Creates.sql and an Inserts.sql which loaded my old data perfectly. This is the only distribution specific (MySQL in this case) step I used, so if you’re using Postgres or something, tackle this in some other way and move on.

One important step: if you have any table names which will conflict with your current django project, rename them before exporting.

2) Prepping Django Models

Load your data into a fresh temporary table, ‘project_temp’ for now. Then create a new django project and hook it up to this temporary table. Don’t syncdb or anything, simply run django manage.py inspectdb > models.py

Then create a new app ‘migration’ in your project (the real one, not this temporary one) and add it to your INSTALLED_APPS. Throw this new models.py into the migration app and then in SQL copy all of the tables from your temporary database to your django project. I didn’t have any conflicting table names but you might. If so, you should have renamed them at step 1.

If your old project has a user table, add a new field to the model called ‘live’, which is a null=True,blank=True FK to your current Django User model. Don’t forget to manually add this field to your old data’s user table.

3) Migration app

In your migration app, create migrate.py–this is the script you will use for looping through the old models and loading them into the new. You may want to break it up into several scripts, as it will probably get somewhat long. Open up that file and add the following to the top:

import sys
sys.path.append(’/path/to/your/projects/’)
import os
os.environ[’DJANGO_SETTINGS_MODULE’] = ‘myproject.settings’
from myproject.migration.models import *

4) Migration scripts

Now that your migration app is setup with all the files necessary, you’re ready to start writing your migration script. The benefit of using Django rather than SQL to prepare is that you can be absolutely sure that your old data is valid against your new models.

In general, I looped through my users, saving them using get_or_create, and then updated my old user table’s ‘link’ FK to point to the newly created user. Then I looped through the rest of my models. Whenever I encountered a reference to a user’s former id, it was easy to hook up because of the link between the new and old user models. I used get_in_create in every case. This makes it so you can stop and restart your script without adding duplicate info to your database. And if your script gets interrupted, any data that didn’t get added will on the next run.

Keep running and rerunning your migrate script, fixing script logic and cleaning your old data until you’ve got a complete database.

Sent the following letter to Apple. Self-explanatory, really.

In my opinion, this is an instance of a more general habit of Apple and other companies of providing restrictive updates without notice and refusing to provide a roll-back.

I’ve heard the argument that they can’t support a 10 year old game forever, however, they’re still selling the game and advertising its compatibility with “OS X”. If they dropped compatibility in OS 11, I would be miffed but wouldn’t mind (I’ll have SC2 to play then, anyway). If the boneheads at Microsoft can maintain support for this game, Apple has no excuse.

Hello there,

I am very disappointed that the 10.5.3 update breaks compatibility with StarCraft. Considering this game is still for sale both online and in stores, this makes little sense.

Also, I am disappointed that no word has been given to the gaming community about this issue. Blizzard, the manufacturer of StarCraft, responded within hours, telling players to post their hardware specs, unfortunately they claim they can do nothing about it and that it is an issue with NVIDIA drivers.

I, for one, have purchased my last Apple product. My iPod recently broke after 5 years of wonderful service, however, I will not be replacing it with an Apple product. I also purchased an iPhone and have since returned it (if you’d like me to provide a receipt proving this I will be happy to oblige). I will simply not be a customer anymore. I’ve purchased two iBooks and one MacBook. I’ve purchased 3 iPods (for myself and as gifts). Receipts can be provided for any of this.

Even a company like Microsoft would never behave in this fashion. Sure, they’ve discontinued support for some hardware in Vista, however, users are given the option not to move to Vista. I was doing what I thought was a routine update, little did I know that support would be removed from my favorite past-time hobby.

My only option at this point is to roll-back to the disk version of Leopard, which I suppose I will have to do. Had support for Starcraft been removed from my Macbook, or had I been told that support would drop within months, I simply would not have purchased it. Plain and simple.

I always hate seeing an error message while loading JSON fixtures while running syncdb after making some changes to my tables.

One major problem is that the table and field of the fixture isn’t outputted in the event of an error. Even specifying the verbosity=2 doesn’t help at all. How worthless is the following error?

yeago@tryonia:~/dev/projekt-x$ python manage.py syncdb
Installing json fixture 'initial_data' from '/home/yeago/dev/projekt-x/../projekt-x/consumers/fixtures'.
Problem installing fixture '/home/yeago/dev/projekt-x/../projekt-x/consumers/fixtures/initial_data.json': [u'This value must be either True or False.']

This value?! What value?! What field!?

Solution

1) vim /path/to/django/db/models/fields/__init__.py

2) scan for the error message (”This value must be either True or False” in my case)

3) before the ‘raise’ statement. import pdb then do pdb.set_trace(). Viola!

yeago@tryonia:~/dev/projekt-x$ python manage.py syncdb
Installing json fixture 'initial_data' from '/home/yeago/dev/projekt-x/../projekt-x/consumers/fixtures'.
> /usr/lib/python2.4/site-packages/django/db/models/fields/__init__.py(489)to_python()
-> raise validators.ValidationError, _("This value must be either True or False.")
(Pdb) self
<django.db.models.fields.BooleanField object at 0x409d3d6c>
(Pdb) self.verbose_name
'vampire victim recent'
(Pdb) fuck yeah!
*** SyntaxError: invalid syntax (, line 1)

Sorry for the exclusive title. This covers not only programmers but the IT sector in general: designers, you too.

This ‘recession’ of ours is interesting because, as far as I can tell the impact simply has not hit us. In fact, my personal experience tells me the opposite. I know a rascally pack of individuals (a design team) who recently quit their day jobs, and are floating without much problem at all. Strange isn’t it that we’re supposedly in a recession? But not if you consider some things.

Competition isn’t terrible for American programmers

Even with the supposed ‘influx of indian computer programmers’ that people liked to go and on about, I’ve never lost a single job to Asia (India and some ex-Soviet bloc). In fact, even given the difference in hourly-cost between Asian and American web developers, competition is still rather strong. I’ve taken on several freelance clients who had previously hired developers from other countries with weaker currencies. On the flip side, I’ve also attempted to outsource work to these programmers and its never paid off much compared to doing it myself.

I believe that new interfacing methods–and thus familiarity and capacity in utilizing them–are simply more natively part of American culture. This isn’t some racist allegation, its just that the devices we sell here, their deeper integration into our lives, and the peculiar lingo which development clusters adopt are simply more centered around English and America.

Some examples include the Django Framework, named after a foreign albeit Americanized musician, Prototype.js (which attempts to incorporate a more natural English rhythm to common JavaScript techniques), Ruby on Rails which is an attempt in the vein of Prototype but for server-side development and even lends its name to an American contribution to technology (yes, Obviously trains are known worldwide by now, that’s not what I am saying. I am simply saying they are at a basic level Americana-centric). Also, all the most active IRC channels and web forums are English.

The Falling Dollar

Another reason that things aren’t terrible here is because countries like the UK are finding themselves with plenty of cash to spend in America, and this is translating to an influx of jobs–new web applications are still a good investment. This is very evident in markets like craigslist. So while many Americans are feeling high tide coming, the IT sector–which for the past decade has been synonymous with inflated incomes–is still finding new opportunities. There is currently lots of talk about the Microsoft-Yahoo alleged buy-out, but let’s not be so quick to blame this on the recession. Yahoo has been a sick dog for longer than that. What about the tech-darling Google? Its stock dropped 200 bucks! But where are their layoffs if things are so bad? Just compare GOOG with the DOW average [hope that link works] and you’ll realize even with their depreciated stock they are still better than the average.

Intra-American Competition

There is also no evidence that a recession has turned up the heat on intra-American competition, either. While one reason is because, as I mentioned above, the pool for demand is growing in areas like the UK, I also suspect that American IT workers are constantly developing new methods and learning to raise their efficiency relative to workers in other sectors. Also, I believe that companies realize that even in times of slow-growth there is a cost-saving advantage to maintaining a budget for IT services. I know my company does.

I won’t get too much into my reasons why IT workers have been able to raise their productivity. We’ve all heard the rant about open source methods. I will simply say that open web forums and IRC channels, as well as a boom in GPL web-frameworks are technically impervious to pitting people against each other within their sector because they are all ultimately in collaboration. In terms of the sharing of methods they are open mediums. Closed, competitive guilds simply can’t form in this environment. Workers in this sector enjoy an immediate and costless resource of each other’s support.

What is Museek (plus)?

Museek plus is a Soulseek client. It is far superior to other products available on OS X in terms of usability, coherence, and ability to specify preferences. Read more on the site.

Unfortunately, its a bit difficult to get working on OS X because its native to Linux. Nevertheless, it can be done (and is well worth it.

Installing: a bit of a pain

With the help of Daelstorm I was able to get Museek working on my MacBook (intel), Leopard (OS X v 10.5). This should be a great relief to any Mac Users still stuck in using Nicotine, or god forbid SolarSeek–I’d rather use BearShare!! (oh, p2p humor… meh.)

Museeq (client)

So here goes:

Firstly, get X11 / MacPorts/ XCode Tools installed on your machine. Takes a while. A long while.

Then run sudo port install qt3 in an X11 terminal.

Also, get Fink installed. At the time of this writing I had to use this build but that will probably change, as Leopard apps are in rapid development all over.

You’re going to have to add the fink stuff to your PATH (.bash_profile file) so you can run its packages from the command line. I believe I was prompted, at the end of the install, to run a little shell script which added the appropriate lines. At any rate, my .bash_profile looks like:

yeagos-magic-machine:museeq y$ cat ~/.bash_profile
export PATH=${PATH}:~/Programming/tools #You don’t need this line
export PATH=${PATH}:/usr/local/bin
export PATH=${PATH}:/opt/local/bin
export PATH=${PATH}:/opt/local/sbin
test -r /sw/bin/init.sh && . /sw/bin/init.sh

I hear the order of the things above counts, but I can’t tell you why.

Also, get the latest version of Museek. And glance over this page because you’re going to have to install a few more things mentioned there.

Daelstorm mentioned a Museeq QT4 port but I didn’t do that.

Now, we will be using the SCons method to build this program with the following command scons QTDIR=/opt/local/lib/qt3 from the folder created by the Museeq tarball you downloaded earlier, pointing to the installation of QT3 you (hopefully) already installed. Oh, and get scons! (fink install scons)

This will probably fail due to missing dependencies. I got choked up on ‘libxml++2′ and ‘vorbisfile’. That’s where fink came in handy with fink install libxml++2 and fink install vorbistools libvorbis0 libvorbis0-shlibs (I’m honestly not sure *which* of these applied to my situation. Try them one at a time if you feel so inclined (and let me know >=] )

Anyway, I got myself into serious trouble at this point.

I started getting errors related to the trayicons one | two

running scons with MUSEEQTRAYICON=no and 0 did not help.

Daelstorm came up with a fix: edit /museeq/SConscript and remove the line “trayicon.cpp” Don’t do this if you are stuck at something that has nothing to do with trayicon / trayicon.o.

Once I did this everything worked fine and I got the wonderful message from scons “done building targets”. When this works, you’ve got a working installation of Museek, except its in the /workdir. To build a global copy at this point you simply go sudo scons install QTDIR=/opt/local/lib/qt3 MUSEEQTRAYICON=0 from the /originalMuseeqTarball/museeq directory and, viola, you have a working copy of Museeq!

If you had the trayicon problem, simply select a theme and restart X11 and your icons are fine!

Museekd (daemon)

Installs as described in INSTALL.scons

At the beginning of my Christmas vacation I got the bad hankering to buy a new car. It was prompted by the usual irrational reasons: I got raise, my current car’s battery just died, mine has high road-noise, etc. During my 4-day temporary insanity, I sat across from many salesmen and wrangled to the point of having a dry mouth several times. During these high-tension sessions, I learned many things about the car buying process.

Leave your checkbook at home

The car-buying process is never a one-day thing. Unless you are either desperate, a fool, or dominated by your impulses. There is no offer not worth sleeping on–ever. Ever ever. Do not let anything–even a broken car–motivate your buying process or you are doomed from the beginning. When you free yourself of this baggage, the car-purchasing process becomes relaxed and fun. Car salesman, like the bloodsharks they are, will pick up on these excited or desperate impulses and give you a good financial rogering. Say to yourself “I will sleep on any offer.”

Find a Favorite Car

When you find the car you really want, all other cars become the comparison. When one salesmen asks you why their price isn’t right, you say “because for only $400 more I can go down the street and get a…”. Make them work to make this your favorite. Meanwhile, make sure the dealership that has your favorite car is aware you are 1) motivated 2)serious 3)interested 4)looking elsewhere until their numbers are right.

Never give a budget

The first thing a salesman will try to do is get you to give your idea of “what you’re looking to pay” for this car. They are just trying to wrench you for a number. Don’t give it to them. In the car market, as well as many other markets, naming the price is the burden of the seller.

What price are you willing to pay for the car? The lowest price they are willing to move it off the lot for. Tell them this and watch the pain in their face. They will try to mask the question by asking you “what your budget is”. Kindly tell them that if the car wasn’t in your budget, you wouldn’t be looking at it.

Don’t waste your time hussling the first smiling-face

Typically, the first car-salesman you meet is not authorized to make really good deals, so don’t worry about talking him down. You do not need to hussle him. You do not need to charm him. He is a chauffeur, nothing more. Do not waste your energy trying to wrestle him down. If you play your cards right, you will go through 1 or even 2 levels of big-bossmen. These are “Sales Directors” and they have way more muscle when dealing with the the accountants (the little room the salesmen disappear to). Keep in mind, if you are at a smaller dealership this may not necessarily be true.

Do not mention a trade-in, student discounts, etc

Car salesman will always ask you about your trade-in early on because they know its one way to confuse your purchase. Don’t mention a trade-in until they are talking a number that is in your range. Trade-in price is always a fake number, anyway–there is a bottom line, and your job is to find it out.

Keep Yer Eye On The Bottom Line

This is the most important section, so its lengthy. Do not lose sight of the bottom line. A car-dealer will always want to talk about your “target monthly payment”, “trade-in value”, “leasing options”, or a financing deal. But they never want to talk about the bottom-line because that’s the only thing that matters and they know it. Dealers will quickly move away from discussion of lowering the bottom-line price and, like the common optical illusion, warp a square into a tall, thin rectangle and try to get you to believe its a different size. Its not. Once you lose sight of the bottom line you are doomed. You need to figure out what your budget is beforehand. If you haven’t done that, stop wasting your time and my bandwidth reading this (ok, well, the page already loaded, but don’t refresh!).

Edmunds Auto Loan Calculator is the best way to figure out what your bottom line is because it accounts for cash-back (which you pay tax on!). You do not sign a deal without plugging the numbers into this calculator. CAPICHE? Bring a notebook and a friend with you, and make them write down every number the dealer offers. You will find out interesting things when you plug them in.

Car dealer: “All you have to do to get your payment down to $275/month is to go with the 66 month lease instead of the 60.”

Logic: 6 more months of a loan * $275 = over $1,600 bucks!

Sure, its elementary math. But when you are sitting across the desk and the sweat is starting to pour, this is easy stuff to lose track of.

Misc

This is a write-up of my trials regarding my 12” G4 iBook.

It all began when a piece of envelope was sticking in front of my slot-loading combodrive and I loaded it. The paper stayed and the mechanics of the drive became screwed up.

I purchased a new combodrive at PowerBookMedic.com. I opened it up and put it in but it didn’t work. But the semester was upon me and I didn’t have time to open it back up to find out why it didn’t work and I couldn’t risk not having a computer. So I set it aside until Graduation. What I found peculiar was while the device made no noise when I powered-up, it showed up in my System Profiler. This I found odd.

Cracking the shell

There are instructions online for disassembling your iBook. I recommend you google around for any of these before and while you are doing so, if you have another computer handy. As for me, I didn’t. I simply gathered some tools: a small philips screwdrive, a small flat screwdriver, and a chisel for prying open the case, and some forceps or tweezers for removing wire-connectors such as those for the speakers, etc. I keep a few rare earth magnets handy as screw-holders. I tend to end up with (no kidding) about 10 spare screws whenever I take apart an iBook.

A mis-connected cable

Some folks on freenode told me that the combodrive jumpers were not set correctly, or that there was otherwise an error with the device and to send it back to powerbookmedic. As with most first-advice at IRC, it was wrong. I called powerbookmedic and they said to check the cable connecting the combodrive, because even a hair-line fracture in it could make it non-functional yet still show up. Sure enough, I opened my iBook and pushed the cable in a bit more and it worked great.

For the first time in nearly a year my computer had a working combodrive in it, and I was excited to be able to play StarCraft again (haven’t been able to run it from an image, for some reason).

Then the bad news

I went out to dinner, thrilled at my accomplishment of being able to repair my iBook myself. I came back to a very strange smell in the room, and a completely dead iBook. When I left, I opened the case and let iTunes run so that some FTP’ing I was doing would not get interrupted. Eventually it turned out the strange smell was coming from my iBook. I was crushed–I realized that I left my iBook sitting on the towel I had used while working on it–I am not sure if this is what caused the diode to fry, but its my best guess so far.

I cracked it open to discover a burned out diode labelled Q81–more or less near the Hard Drive. (Picture). Folks at IRC are saying that I should probably just order another motherboard.

My roomate was having a problem with files not showing up in spotlight or finder. Files would show up once they had been opened but lots of old files would simply not appear—even if you opened a folder and searched it for a file you were staring at. I checked spotlight preferences and there was nothing wrong.

People at this forum suggested rebuilding the index.

This page suggests rebuilding it with the following command:

sudo mdutil -E /

But that didn’t work. That page also suggests downloading OnyX, a general utility and maintenance program and clearing the Font Cache and other repaire procedures. That didn’t work either.

At this point I didn’t know what else to suggest besides reinstalling OS X. But then I just realized you can turn indexing off:

sudo mdutil -i off

Viola. This will slow down searching, but at least its not using a funky, broken index.

What Not To Do

Salon.com’s review of No Country For Old Men is surprisingly terrible. Not quite as bad as this one, but they’re definitely in the same ballpark.

Firstly, if you’re going to bother reviewing a movie, please attempt cover more material than you found while scrolling the IMDB page. Please give us something more than stock praise for things like its (immaculate) cinematography and the already-token guffawing at Anton Chigurh’s hair (which, I’m sorry to inform you all, makes him more badass, not less).

Secondly, in order to review a movie you have to rise to an above-average understanding. Having a dim idea just won’t do, but if that’s the best you’ve got, at least try to cover that fact up. Don’t say things like:

“No Country for Old Men” is a thriller, although the plot points often connect only tangentially. It’s not supposed to add up in the end, and we’re not supposed to read its plot twists and turns all that literally.
—Stephanie Zacharek @ Salon

Hmpf. So you watched the movie and really found it to be a disjointed David Lynch mish-mash? Then blog not. No Country was exactly opposite of this fumbling description—I’m hard-pressed to think of a nuance in this movie that wasn’t thoughtfully placed, and deeply entrenched in the plot lines. Typically, a slightly off-cuff movie will bathe in vagueness in order to remain entact—this movie didn’t. Every plot point was neatly tied up by the end, only you had to pay attention. Every scene that seemed to connect “only tangentially”, Stephanie, was probably answering questions you just didn’t think to ask.

Salon.com also illiterately refers to Anton’s weapon as “a pressurized thingie used to stun and instantly kill cattle”. I’m not going to pretend to know the name for this ‘thingie’ (aside from air gun and maybe cattle gun), but if you’re going to blog about it, do some google footwork, eh? Provide a service.

Ok then, add something.

I’m going to go out on a limb and say the opposite of everything I’ve read in every other review: this movie is not a thriller. Whaa?? Ok, it was half-thriller—which explains not only the introduction, conclusion, and a dozen or so vigniettes in between (not to mention the title No Country For Old Men). It was a thriller wrapped in the frame of an old-timey piece. We the audience are told why this isn’t the country for old men, but the movie makes a clear priority of these men in the very vivid portraitures of older gentlemen characters—first and foremost Tommy Lee Jones. To see this story as a thriller for your own eyes, rather than seeing it from theirs is, in my opinion, to miss half of the film.

I have sympathy for those who were confused the first time you saw this movie (unless you blogged about it). This movie did seem disjointed but I promise it answers most of its questions (I’m going to comment here some questions I had). Also, I’ll try not to spoil the plot but this movie is daring with its expenditure of characters—the ‘protagonist switch’ that happened late in the movie at first view took me by surprise (until I watched it again and realized this movie was not about Anton -> Llewelyn feud).

In a hilarious and gut-wrenching way my friend’s cell phone went off for a minute in her purse during the closing sequence, as Sheriff Tommy Lee is explaining his dream. I’m glad I saw the end again—it was simple and genuine in a way I’m sure many viewers scoffed at as misplaced or contrived. Meh, I guess most of us are cynical, stimulation junkies. And here we have a movie about exactly that.

The question

Dear Sir or Madam at Wholefoods Market,

We am writing to you about your 365 brand of All Natural Sandwich Cremes. Firstly, let us say that I enjoy them immensely. They make a wonderful late-night snack and the ladies love them.

Now to the question. We are wondering why, when the cookie is separated, does the creme seem to prefer the vanilla half. When I say prefer, I mean that every time I can remember the creme has ended up on that side. We have a theory, although we’re not sure if its correct. I would share it, but I don’t want someone to just reply with ‘yeah, you’re right.’ We actually want to know.

Thank you for your time,

Steve and Kim

The reply

Thank you so much for your patience. I have done some research and can not uncover any reason for this. No one is really sure why that happens. I’m sorry I don’t have a concrete answer for you. Let me know if you have any more questions.

Jason Hays
Guest Services Content Administrator, Private Label

But this was not satisfying…

Subject: Re: cookies

Can you please tell us which side the creme is first applied to by the machine, during production? Is it always the chocolate, always the vanilla, or both?

Steve and Kim

And the conclusion…

Steve,

Thanks for the email. I called the manufacturer and the crème is added to the vanilla side (most of the time) and the chocolate side is added to complete the sandwich.

If you have any further questions, please let me know.

Jason Hays
Guest Services Content Administrator, Private Label

The hypothesis holds!

When the machine puts the creme on the vanilla half, the creme is hotter. As it cools, stronger molecular bonds form so when the chocolate side is put on, its already a bit cooler. No such bonds form.