Greg [entries|archive|friends|userinfo]
Greg

[ website | gregstoll.com ]
[ userinfo | livejournal userinfo ]
[ archive | journal archive ]

Links
[Links:| * Pictures * LJBackup * Where's lunch? * iTunesAnalysis * del.icio.us bookmarks * My google map * Twitter ]

link friday! [Jul. 10th, 2009|09:58 am]
[Tags|, , ]
[Current Mood | groggy]

And since it's 8(!) days until the wedding, presumably my last one as a swinging bachelor.

This fascinating NY Times graphic shows the various business cycles and the recession we're in now. And maybe some hope that things will be better in 6 months.

A 60 foot Gundam robot has risen in Tokyo. I really really hope this is real.

Short article on the 40th anniversary of Stonewall. Found this surprising:
In 1966, three years before Stonewall, Time, then the voice of middlebrow, middle-class respectability, published a long essay on “The Homosexual in America.” The magazine, while acknowledging that “homosexuals are present in every walk of life,” concluded that homosexuality
is a pathetic little second-rate substitute for reality, a pitiable flight from life. As such it deserves fairness, compassion, understanding and, when possible, treatment. But it deserves no encouragement, no glamorization, no rationalization, no fake status as minority martyrdom, no sophistry about simple differences in taste—and, above all, no pretense that it is anything but a pernicious sickness.

Sacha Baron Cohen's (Borat, Ali G, etc.) new movie Bruno comes out today- this character is a gay fashion person or something. He went out of character on Letterman describing how he interviewed a terrorist. Salon thinks it's bad for the gays, Slate thinks it's good for the gays, and Ebert surprised me by liking the movie.

Ricky Gervais's The Invention of Lying looks like a good movie, and it has "every other Hollywood actor and comic in the world" (Patrick Stewart! Tina Fey! Jason Bateman and Jeffrey Tambor! John Hodgman!)

Cool jQuery plugin to dynamically make nice-looking graphs - uses the new canvas feature, like my marriage map does. Wish I had thought of this!

California v. Texas - take that, California! Although
Despite all this, it still seems too early to cede America’s future to the Lone Star state. To begin with, that lean Texan model has its own problems. It has not invested enough in education, and many experts rightly worry about a “lost generation” of mostly Hispanic Texans with insufficient skills for the demands of the knowledge economy. Now immigration is likely to reconvert Texas from Republican red to Democratic blue; Latinos may justly demand a bigger, more “Californian” state to educate them and provide them with decent health care.
Link2 comments|Leave a comment

Optimizing iTunesAnalysis: faster database access [Jul. 9th, 2009|09:44 am]
[Tags|, , , ]
[Current Mood | tired]

The second in an occasional series
Idea: Now I'm focusing on improving the time to insert the iTunes data into database. Where we left off last time, our script took 71 seconds to run, ~50 seconds of which was database operations. The idea I had to speed this up was to batch a bunch of queries together and thus make fewer calls to the database. It turns out this actually slowed things down.

So I did a little research and it turns out if you insert data with the same query structure over and over again (but with different bind variables), the database doesn't have to reparse the query which speeds things up a lot. I tried doing this with pyPgSql but couldn't find any documentation how it was supposed to work, so I switched to using psycopg2 and changed the query for inserting the playlist data. Just switching to a psycopg2 sped things up a lot, it seems. I tried switching to a similar sort of query for inserting track data, but that actually slowed things down.

Anyway, the new script runs in 25 seconds, and it looks like only around 9 seconds for database operations. This is a 400% speedup in the database time! Overall, this step improved performance by ~180%, and since we started at 114 seconds we've improved ~350%.

Conclusion: Another big success, and I'm not sure how much more I can squeeze out of the iTunesInfo.py script. Next time I'll focus on the analyzedb.py script, which does the analysis from the database - right now it's taking between 5 and 12 minutes to run on my library of 6400 tracks.

Source files:
- old script
- new script
LinkLeave a comment

Optimizing iTunesAnalysis through smarter parsing [Jul. 8th, 2009|10:31 am]
[Tags|, , , ]
[Current Mood | geeky]

The first in an occasional series
Intro: A while back I wrote a script to analyze an iTunes library and find your favorite artists, albums, etc. It works pretty well and I regularly use it to update my own analysis. Unfortunately, it generally takes a long time to run, which is sort of OK for me (because I just start it running and go do something else) but less good for people who are running the analysis through the web site.

So I'd like to make it run faster, and I have a number of ideas to do so.

Idea: There are two main parts to the system - parsing the iTunes Music Library.xml file into a database, and running the analysis on the database. First I'm focusing on the parsing part.

The first version of the parsing script uses Python's xml.dom.minidom package to completely parse the library file.

After profiling the first version by running python -m cProfile -o profiledata.oldway iTunesInfo.py "iTunes Music Library.xml", I see that the whole parsing process takes 114 seconds. The major parts of this are 60 seconds for the xml.dom.minidom.parse method and 46 seconds for the database operations. Note that this only leaves ~8 seconds for figuring out the track information - clearly this is not the bottleneck!

So I'd like to improve parsing speed. There are two basic kinds of XML parsers - what we're using now is a DOM or Document Object Model-style parser, which means that the parser reads the entire file in and returns a parsed structure containing all the data. (I remember writing a simple XML parser that did this as a project in COMP 314. Ah, memories...) The advantage to this method is that after the parsing is done, it's easy to traverse the DOM tree and find the data that you're interested in. The downside is that, well, it's slow. Also, the entire document has to be read into memory which means that your memory usage is proportional to the size of the file you're processing, which adds to the slowness and can lead to out of memory problems on huge files (although we weren't seeing that here).

The other basic kind of XML parser is known as SAX, or Simple API for XML. You provide callback functions that are called whenever the parser runs across the start of a tag, end of a tag, character data, and...that's it. Whatever processing you want to do you have to do in those callback functions. So if you're just, say, counting the number of <key> tags in a document this works really well. It's also much faster than the DOM-style parser, since it doesn't have to generate a giant tree structure. But doing the sorts of processing we're doing on the library file seems a bit more tricky.

Anyway, I take a stab at it, and after a bit end up with version 2 of the script. Notice that the logic in the Handler class is a bit twisted - we have to keep track of where we are in the document (so if things get out of order we'll have problems) and use a state-based system which is a bit brittle and unclear.

But how does it perform? The old version of the script ran in 114 seconds, and this version runs in 71 seconds for a ~60% increase in speed. But really, it's better than that, because the database operations still take around 50 seconds - if we subtract that from both we get 64 seconds versus 21 seconds which is a ~200% increase in the speed of the parsing.

Conclusion: This was a big success! Most of the time is now in the database layer, which I have some ideas for speeding up next time.

Source files:
- old script
- new script
LinkLeave a comment

weekend, life update [Jul. 6th, 2009|01:08 pm]
[Tags|, , ]
[Current Mood | okay]
[Current Music |Nine Inch Nails - "The Perfect Drug"]

Thursday I went tubing with some people in New Braunfels. I'm generally not super excited about water activities, but it was fun and relaxing. It was also really, really hot. I didn't fully reapply suntan lotion while floating down the river (not really sure why I didn't, except that I didn't feel like I was burning) , and thus I ended up horribly sunburned on most of my chest, upper legs and feet.

I've been using some aloe lotiony stuff which helped a lot, but even still it's still sensitive and (worst of all) itchy, especially at night. Saturday night I took a Benadryl which successfully knocked me out, and last night I tried to get by with just some cortisone cream. Gave up on that after a while of itching and not sleeping and took the Benadryl, but it didn't work as well...got to sleep after about an hour and didn't sleep well. (neither did [info]djedi, for unrelated reasons)

12 days until our wedding! Everything seems in order and we're keeping up with our todo list but I'm still generally stressed. Picked up honeymoon tickets, etc. today and I'm looking forward to that part :-)

A somewhat rambling but interesting talk by Stephen Fry about America's place in the world.

Propaganda posters for World War III. I think this one is my favorite.
Link2 comments|Leave a comment

happy 9 years! [Jun. 30th, 2009|10:22 am]
[Tags|]
[Current Mood | cheerful]

And on the same day of my 1000th LJ post! Clearly this is an auspicious day. Except for the fact that I broke a glass this morning :-(

Firefox 3.5 is out today! Looks like it's not automatically updating yet. I've been beta testing for a while and it's speedier and more responsive than Firefox 3. You can see some of the new features here.

I'm becoming somewhat obsessed with the Palm Pre - been reading reviews from Engadget and Gizmodo (here's the flash-heavy Sprint page) and I'm kinda excited about it, although I'm under contract with T-Mobile until March. I even signed up to develop an app for it - gonna try porting whereslunch.org to it and see how easy it is...

Sometimes I step back and marvel that I can buy 16 GB of storage that fits on my keychain for $35.

Internet is out at home until Friday.
Link1 comment|Leave a comment

Games database [Jun. 27th, 2009|08:20 pm]
[Tags|, ]
[Current Mood | cheerful]

Just got done working on a database of the games we own. Not everything we own is in there, but most of it is, and I'll try to get the rest added soon. You can search based on number of players and the length of the game. Give it a shot and let me know what you think!
Link7 comments|Leave a comment

linky in spirit [Jun. 26th, 2009|10:58 am]
[Tags|, , , ]
[Current Mood | cheerful]
[Current Music |Michael Jackson - "Black or White"]

We watched Dial M for Murder last night. I had high hopes for it, having enjoyed Hitchcock's "North by Northwest" a few weeks ago, but this was even better! The opening 15 minutes or so were packed with tension, and although sometimes older movies don't hold up today since their twists have become terribly commonplace, this one holds up quite well. Highly recommended.

The Alamo Drafthouse had a tribute Michael Jackson singalong last night, and they say there will be more this weekend. I went to one of these in 2005 and it was a lot of fun.

Supreme Court rulings: strip-searching a 13 year old because you think they have Advil = very not OK (Clarence Thomas was the lone dissenter), and crime laboratory technicians must testify to admit lab results into evidence, which sounds like it might be a fairly large change. That was a 5-4 ruling with an odd majority: Stevens, Souter, Ginsburg, Scalia and Thomas.

If you're interested in the Supreme Court (and who isn't?), here's a chart of the justices' ideology over its history, which explains why seeing Stevens and Thomas agree on anything is pretty weird.

The DNC gay fundraiser I mentioned last week happened, and apparently Joe Biden gave a good speech and got a lot of applause. But it's hard to read this:
He said that gay and lesbian concerns will not be "delayed, put off or not end up on [Obama's] plate" because he is dealing with so many other issues.
since that seems to be exactly what's happening. I appreciate that they're pledging to repeal Don't Ask Don't Tell and the Defense of Marriage Act, but talk is cheap.

For you non-Austinites: it hit 107 degrees yesterday at the airport, and it's supposed to get up to 106 today. It is very very hot. Pity us!
LinkLeave a comment

craving HDL [Jun. 18th, 2009|09:31 am]
[Tags|, ]
[Current Mood | excited]

I had a physical yesterday, and like last time my HDL is still low. (it's 28...somethings and normal is 40...somethings) The good news is that my triglycerides and LDL were normal this time. Oh, and I need to lose some weight, which I already knew.

I got my results on a nice little sheet with the normal values and what I could do (non-medicationwise) to improve the values. On literally every measure, the first two items were "Lose weight" and "Exercise more". Um, great.

Also, I moved at work today! Exciting.
Link1 comment|Leave a comment

teh gays !<3 Obama [Jun. 17th, 2009|04:08 pm]
[Tags|, , ]
[Current Mood | irritated]

Last week the Department of Justice filed a brief defending the Defense of Marriage Act. Now, DOMA is the law and the Justice Department is required to uphold the law (although apparently previous DoJ's have published opinions saying they thought laws were unconstitutional), but the brief itself is pretty terrible - saying DOMA is consistent with equal protection, comparing it to laws banning incest, saying the right to marry isn't fundamental when it comes to marrying someone of the same sex. And according to Robert Gibbs (the press secretary) Obama stands behind the brief

So. This pissed a lot of people off. There's been a lot of unhappiness already about the lack of movement on DOMA and Don't Ask Don't Tell, and even the HRC published a kinda mad letter. A lot of people are pulling out of a DNC gay fundraiser next week. People generally feel betrayed.

Presumably to address these issues, Obama announced the he would grant same-sex benefits to federal employees, which is great except that said benefits don't include health insurance. Although it does include relocation assistance. Which is, ahem, something.

Anyway! Rant over.

A story of homeless Sims in The Sims 3. The narrative is pretty touching.

GameFAQs recently ran a Best Game Ever bracket, which Zelda: Ocarina of Time won. I was happy I've actually played a lot of the games in the final bracket!
Link2 comments|Leave a comment

filllllings...nothing more than fillllllings... [Jun. 16th, 2009|10:04 am]
[Tags|, , ]
[Current Mood | happy]

I went back to my new dentist at Shoal Creek Dental Care - the cleaning I had took a while but was much more pleasant than usual, and I got a tooth filled yesterday in 20 minutes, which is awesome. And my cheek/gums didn't hurt after the numbing wore off! I am quite pleased.

Courtesy of FiveThirtyEight, I found this awesome chart:

Some interesting things:
- Housing antidiscrimination is the most popular policy in all 50 states, but it's only been enacted in 20 states. My guess is that it's something that people don't run into that often (we've looked for housing in MD and TX and never felt discriminated against) so there's not much impetus to pass it.
- Bully for Iowa's Supreme Court, but I'm worried about marriage being overturned there.
- Obvious next targets for marriage: NY, RI, and CA (oh the irony!)
Link6 comments|Leave a comment

better living through science [Jun. 12th, 2009|03:32 pm]
[Tags|, ]
[Current Mood | geeky]
[Current Music |that "Jai Ho" song]

A while back I wrote a script to analyze our GnuCash account. A few months ago, I added a quick little feature to predict how much money we're going to spend this month - basically just taking the existing values and extrapolating to the end of the month, disregarding stuff that is largely the same like the mortgage payment and bills. So every time I run the script I get an updated projection of how much money we'll spend this month.

For something so simple, it's surprising how it's changed my behavior. I'm very much more conscious about spending money now, and last month we spent the least amount we have in a while. Of course, that's just one data point. It's similar as the idea behind the Google PowerMeter - if you can measure something well you're more likely to think about it more and conserve when you can.

In a similar vein, the Wall Street Journal writes about not getting enough sleep and a device called the Zeo which tells you how well you slept the next morning. Again, the idea is that you can try out different things (cutting out caffeine longer before bedtime, etc.) and see how they affect your sleep patterns. Cool stuff! Yet another study shows that sleep is important and helps you learn.
Link2 comments|Leave a comment

just this guy, you know? [Jun. 10th, 2009|11:49 am]
[Tags|, , ]
[Current Mood | busy]

A new poll came out showing that 69% of Americans favor allowing openly gay people in the military - up from 64% 5 years ago...and even 58% conservatives support it! For a point of comparison, when Truman integrated the army only 13% of the population supported it. Stephen Colbert is doing his show from Iraq this week (in front of troops) and last night did a segment on Don't Ask, Don't Tell which was pretty ballsy.

25 Great Calvin and Hobbes Strips

Remember that long health care article? Turns out Obama read it too and is taking it pretty seriously.
Link1 comment|Leave a comment

I'm in a hurry to get things done... [Jun. 5th, 2009|01:34 pm]
[Tags|, ]
[Current Mood | stressed]
[Current Music |Dave Matthews Band - "Funny the Way it is"]

Obama gave a big speech in Cairo yesterday - here's the full text and the hour-long video. Pretty impressive and even-handed.

Yesterday was the 20th anniversary of Tiananmen Square, and the Chinese government tried to block foreign reporters from recording by...using umbrellas.

New Hampshire passed a same-sex marriage bill into law this week, leaving Rhode Island as the only New England state that hasn't legalized it. Which is pretty crazy when I stop and think about it. The map is turning more green!

A long article about gay marriage from a Burkean conservative point of view.

Loving Day, the anniversary of the Supreme Court striking down anti-mixed race marriage laws, is this weekend.

Cool guys don't look at explosions (video).

A long "screenplay" about how hard it is to get security right - this really opened my eyes!

Pixar vs. the Rest, or "Pixar movies, they make a lot of money and people like them".

Stem Cell Contact Lenses Cure Blindness in Less Than a Month - um, wow.
Link8 comments|Leave a comment

other odds and ends [Jun. 2nd, 2009|10:49 am]
[Tags|, , ]
[Current Mood | distracted]

As a wedding present from [info]djedi I got a new monitor! Whereas my old one could fit 4 xterms without overlapping, the new one can fit 9 (and 12 with just a little overlap). This makes a huge difference when I'm programming - being able to see larger sections of the file I'm working on (and having more xterms open in different directories, etc.) lets me get distracted less by switching between virtual desktops, etc. Also, the screensaver of pictures looks much more impressive :-)

We saw "Up" on Saturday (in 3D, no less). It was sweet - the first five or so minutes were really really great. I hope I can be Ellie to David's Carl. Or something. Overall I liked it but less than Wall-E and The Incredibles.

Last night we put in 10 or so attempts on Hodir and got him down to 23% or so. It's pretty overwhelming at first - I have to find the fire and stand near it, avoid icicles, dispel frost novas, heal like crazy through the frozen blows, and not screw up the flash freeze. I got pretty good by the end except at the avoiding icicles while healing through frozen blows. Maybe next time...

If you installed .NET framework 3.5 SP1, you probably got an unwanted silent installation of a Firefox addon that makes it less safe - the link has directions on how to remove it. That kinda sucks.


This whole post is really just an excuse to post this short video which I won't spoil. Also: 100 movie lines in 200 seconds.
Link2 comments|Leave a comment

Yeah, yeah, my heart's in a whirl [Jun. 1st, 2009|11:53 am]
[Tags|, , ]
[Current Mood | okay]

May had some high points, but overall it kinda sucked, what with being stressed out with work stuff all the time. June looks to be somewhat better, but still stressful. July will be hectic for 17 days, then awesome, then more awesome. (August will be extremely stressful for about 4 days, then presumably back to normal) This would all be more managable if I hadn't been so moody. Hoping that goes away when the stress is gone.

This is a terrible, terrible attack on Sotomayor. Something tells me G. Gordon Liddy thinks women shouldn't hold any positions of power anywhere.

Bing is Microsoft's new search engine. It seems decent so far.

Nevada gets domestic partnerships (over the governor's veto) and Illinois's civil union bill makes it out of committee but not to the house floor although it's still possible it will in a special session.
Link1 comment|Leave a comment

days go by and still I think of you [May. 29th, 2009|10:13 am]
[Tags|, , ]
[Current Mood | calm]
[Current Music |Dave Matthews Band - "Dive In"]

Had a nice relaxing week, apart from being sick Tuesday and Wednesday. Oh, and we downed a few new bosses in Ulduar-10 last night (Kologarn and Auriaya), although no loots for me.

Despite being sick most of the week I found some good links:

- Dave Matthews Band has a new album coming out next week, and you can stream the full thing at Pandora. So far it's pretty good!

- This article about health care in the New Yorker was both fascinating and a little depressing. It looks at why McAllen is the most expensive city for health care in the US, and basically the answer is because the culture for doctors there is to try to maximize revenue.

- How an Intern Stole NASA's Moon Rocks

- A rebuttal to those who claim that the new Star Trek movie threw away a bunch of continuity...short version, Star Trek has been throwing away continuity for quite a while now.
Link6 comments|Leave a comment

in the merry month of may [May. 22nd, 2009|02:12 pm]
[Tags|, , ]
[Current Mood | distracted]

My same-sex marriage map got some link love from Metafilter (thanks [info]kernelm!) and somethingawful, oddly enough. And twitter, come to think of it. This added up to 225 hits over the past 3 days, which is a lot for me.

I'm not sure why it makes me so excited to see people linking to my stuff - there are no ads on the page, and presumably most people click on it, take a look, think "Neat!" (or "What a piece of crap!") and move on with their day. I guess it's a measure of fame or prominence, however slight.

The government just released data.gov which is a collection of parseable data from various government agencies. Unfortunately, there isn't a lot of too interesting stuff there yet, but there is a a contest to write a cool app. (the first entrant is FBI Fugitive Concentration!)

The Dallas Cowboy's new stadium has the world's largest HD video screen.
LinkLeave a comment

marriage: the gayening [May. 20th, 2009|02:54 pm]
[Tags|, , ]
[Current Mood | chatty]
[Current Music |Michael Andrews -"Mad World (Alternate Version)"]

The New Hampshire legislature narrowly didn't pass the gay marriage bill (amended as the governor requested), but sent it back to committee so it might come up for a vote again in two weeks. Of course, then who knows if the governor will sign it?

Supposedly the California Prop 8 lawsuit decision will be handed down tomorrow. If it overturns Prop 8 it's possible gay marriage will be legal in California again. Not having followed the proceedings at all I'd bet against it, though.

The Nevada legislature passed a domestic partner bill but the governor has said he'd veto it.

Gay Marriage Slow to Draw an Opposition in N.Y. - good?

All these things are keeping me on my toes updating the same-sex marriage map. Hoping to add a few small features later this week.

Non-gay marriage links:

You know how everyone says the divorce rate is 50%? Apparently that's not true if you look at it as how many people have ever been divorced.

Lost Season 5 recap with crazy crazy theories.

A review of "Glee" which we watched last night. I thought it was kinda (and surprisingly) good!
Link4 comments|Leave a comment

rumsfeld flashbacks [May. 18th, 2009|10:41 am]
[Tags|, , ]
[Current Mood | okay]
[Current Music |"Star Trek" soundtrack]

Apparently the President's Worldwide Intelligence Updates (prepared by the Pentagon) often had verses from the Bible on the cover, which makes me a little queasy. I agree that, if you're religious, there's nothing wrong with looking to your religion/God for faith and guidance, but this seems more like "Hey, the Bible says this whole war thing is A-OK, carry on!" The accompanying article is full of new information about Rumsfeld, including cut for length )
Another excellent article I read this weekend was What Makes Us Happy? in The Atlantic. (a magazine I consistently enjoy) It looks at a study that started following Harvard students in the 1930s and kept up with them until now, trying to determine what factors were most important to living a happy life. What they found was
Employing mature adaptations was one. The others were education, stable marriage, not smoking, not abusing alcohol, some exercise, and healthy weight. Of the 106 Harvard men who had five or six of these factors in their favor at age 50, half ended up at 80 as what Vaillant called “happy-well” and only 7.5 percent as “sad-sick.” Meanwhile, of the men who had three or fewer of the health factors at age 50, none ended up “happy-well” at 80. Even if they had been in adequate physical shape at 50, the men who had three or fewer protective factors were three times as likely to be dead at 80 as those with four or more factors.
(bolding mine) The "mature adaptations" they mention consist of
altruism, humor, anticipation (looking ahead and planning for future discomfort), suppression (a conscious decision to postpone attention to an impulse or conflict, to be addressed in good time), and sublimation (finding outlets for feelings, like putting aggression into sport, or lust into courtship)
(bolding mine)

One more: here's Obama's commencement speech at Notre Dame.
Link4 comments|Leave a comment

hurried links [May. 15th, 2009|02:34 pm]
[Tags|, , ]
[Current Mood | busy]

Is it the end of May yet?

A Roomba's path (by taking a long exposure time picture) is pretty chaotic!

Yes, Star Trek: The Next Generation had a torture episode.

In The Fierce Urgency of Whenever, Andrew Sullivan is pretty pissed Obama hasn't done much for gays yet. I mostly agree, although I'm a bit more patient. It turns out Obama wrote a personal note saying he's "committed to changing our current policy" of Don't Ask, Don't Tell.

The governor of New Hampshire says he'll sign the gay marriage bill if religious protections are added, which the legislature has indicated they'll do. Sweet! For the record, I have absolutely no problem with saying religions don't have to recognize gay marriages. (even though, to my understanding, they don't have to even without the wording he wants in the bill) After all these bills go into effect, Rhode Island will be the only state in New England without gay marriage. That is some craziness, folks.

With The 'Gay Tax,' Love Doesn't Come Cheap - why, despite the awesomeness that is states allowing gay marriage, the fact that it's not recognized at a federal level still costs couples a lot of money.
Link3 comments|Leave a comment

navigation
[ viewing | most recent entries ]
[ go | earlier ]