joelapenna.com/blog

This blog has moved

2010-03-14T18:11:00.001-07:00

This blog is now located at http://blog.joelapenna.com/. You will be automatically redirected in 30 seconds, or you may click here. For feed subscribers, please update your feed subscriptions to http://blog.joelapenna.com/feeds/posts/default.

This Addiction

2010-02-16T23:01:00.000-08:00

Alkaline Trio released Agony and Irony two years ago to much disappointment and sadness amongst my group of friends. The music was too polished, to blatent and well... you can read what I had to say here: http://www.joelapenna.com/blog/2008/07/alkaline-trio-breaks-my-heart

Tts just been a year and a half later but I'm pretty happy with what they've put out. One thing that bothered me most at that time was the divergence between their music and my life; I think I'm OK with that divergence now and I feel like the music this time around is less redicuolus and can more generically apply to listener's lives.

Given the general theme people speak of when describing this album is its throwback to old material much of what I've written here is along those lines.

This Addiction -- The first single off the album feels like it would fit between From Here to Infirmary and Good Mourning. Simple lyrics and riffs but more melodic than their previous efforts.

Dine, Dine My Darling -- Hopeful sounding song typical of Dan's newer songs (like Blue Carolina). "But these drugs are lame..." what? Its really catchy and quick and I do like the word play.

Lead Posioning -- "We're bitter now but better friends." Another catch rhythm guitar and I would love to hear this one on Rock Band. WHAT THE FUCK THERE IS A HORN. I was trying to trace the sound in this song but can't place it. Then I heard the horns. Then I heard the opening riff from "Goodbye Forever." Pretty interesting!

Dead on the Floor -- I'm immediately thinking "Radio," here. "Like two ships in the night we're colliding and sinking into the black seas of our love." -- No vampires, zombies or blood references. See? A bit more subtle. "I got off the plane with my heart soaring now its falling like snow."

The American Scream -- The last time Alk3 wrote a politically charged song, Warbrain I felt the song made little sense. This is a nice attempt but I don't associate politically motivated songs with alk3 but I still give this song a pass.

Off the Map -- "I'll lock it down and we'll make due." A song by Dan misunderstanding a situation and going so far off course everything looks peachy but he still can't tell if things are going to be OK. I'm cool with it.

Draculina -- I didn't like the opening verse but the chorus is pleasant enough but I especially dislike "I'm leaving for draculina, whoa oh oh oh." This is probably the worst track on the album. For poor language and over-harmonizing and whoa-oh-oh-ohs I'll delete it off my computer when I buy the album.

Eating me Alive -- Hi synth. You don't belong here. No thats not true; I like the psychadelic furs and that is who I'm going to have to say the music in this song comes from. I think I might actually like this song a lot. If John hughes were around we could have a cool movie with Alk3 on the soundtrack.

Piss and Vinegar -- "Feels like its raining all the time." I've heard that lyric before and I feel like I've heard this song before... uninteresting.

Dorothy -- Awesome opening riffs that immediately cut to a song that SHOULD have been on From here to Infirmary. So good.

Fine -- A really calm song to close out the albumn and while a song I'll listen to, I'll likely never put it on intentionally.

Overall, I'd say this album is not fast enough to say they've really gone back to their roots but has some really fun songs and they're doing some new things that don't suck which is quite a change from the last album. Then again, maybe I'm just wishing the album is better than it is.

Android Location APIs and why they anger me

2009-11-08T16:52:00.001-08:00

The problem lies in the fact that as a end-developer I have to care about the various location inputs on a phone. I have to think about GPS vs Wifi vs Cell Tower LocationProviders in order to really take advantage of the phone's location capabilities. What basically happened was that my algorithm for picking a users location based on incoming location updates caused more accurate but out of date location updates to be preferred over slightly less accurate but much more recent updates.

This is the code that defines my "best location" strategy. http://code.google.com/p/foursquared/source/browse/main/src/com/joelapenna/foursquared/location/BestLocationListener.java

I basically have to deal with the fact that I might get location-from-gps after I get location-from-wifi-tower OR i might get location-from-wifi-tower after I get location-from-gps so that the Last Known location may in some cases return a more or less accurate location than what the phone is really capable of reporting just because I wanted to be able to get a quick-kind-of-accurate-lock while I was trying to accquire a accurate-as-possible sort of lock.

How would I make this better?

I could see an API where instead of registering with individual location providers, you would query the system for a set of location updates. You would define that you want FINE location, but will accept COARSE updates first. You would have a getBestKnownLocation instead of a getLastKnownLocation that would return a location based on heuristics you might define when registering your location provider. Visually, the updates you'd receive would look something like the growing/shrinking circles you get in the gmaps app when honing in on a location. Maybe the developer would request a "HyperLocalWhileWalkingAround" strategy that could be swapped with "PassiveUpdatesWhenLocationIsSignificentlyChanged." both of which could have knobs for controlling specifics of the strategy.

In short, I want to abstract away the code I have shown as an example above. As a end-developer, I don't care where the location comes from, just that I get a location but I do care, what the data looks like.

Nine days of Nolstaga

2009-11-02T10:31:00.000-08:00

I had a very special nine days in the Midwest. Probably some of the best nine days in the Midwest that I can remember. Things:

Concert with Chris - We've been seeing shows together for almost 10 years, and its been 8 since we first saw The Lawrence Arms opening for Alkaline Trio at the 2001 Halloween show. Just wow.
Zombie Crawl with Chris, Andy, Jared and others - All sorts of DSF/Chicago drinking memories coming back to life.
Week at home with family - Good to see everyone. I got a night out with my brother, mom and dad. Would have been nice to spend some time with Deni, but she was sick and then back at school.
Driving to Michigan with Chris and Football Master - I spent years making trips to Michigan with Chris and many others. Doing it again, despite the 6 hour drive there, was fun -- I can feel you breathing.
Halloween Parties in Michigan - I hope that there are and that there are not photos of the hyjinx that ensued Friday Night to Sunday afternoon. Great times with great people.

I swear that every time I go to the midwest its going to be my last social visit. I don't know when that will ever really happen though. Every time I go, I leave hoping that the next trip will be as good as the trip just ending.

Dutch! Foursquare! Translation! Exclaimations! Are! Overdone!

2009-09-08T20:40:00.000-07:00

I just submitted a change to the foursquare-for-android repository that enables a Dutch translation of our foursquare-for-android app! Neat! Thanks Marco!

Foursquared Repo

2009-08-10T17:00:00.000-07:00

Get the code or file bugs at: http://foursquared.googlecode.com

git

2009-07-21T22:24:00.000-07:00

After spending 2 nights trying to fix my git repository that I broke following instructions on line about removing files from repo history, I'm back to a state where my repo is sane and no longer contains those files I was concerned about. In the course of doing this I had decided to migrate to mercurial, but damn,

rebase --interactive

is very nifty. It allowed me to pretend that the offending commits never even happened! I am still going to switch to mercurial; now that my repo is cleaned up, I should be able to be schizophrenic and switch between which ever VCS I choose

Foursquare Screeny

2009-07-20T11:01:00.000-07:00

Bookmarks Live Folder

2009-07-02T23:21:00.000-07:00

Want a "live folder" for bookmarks on your android home screen?
http://market.android.com/search?q=pname:com.googlecode.livebookmarks

New icon for Foursquared

2009-07-02T00:13:00.000-07:00

Chris made me an awesome icon for the app. In return he got an alpha release! How lucky is that! Now for the low low cost of a line of code or cool menu icons, you too can have some buggy android software!

Android, Foursquare and Push Notifications

2009-07-01T23:02:00.000-07:00

How to get push notifications on Android.

As we all are aware its possible to run apps in the background in Android. The way push notifications work for the google apps like gmail and gtalk are by way of TCP connections with long TTLs. This allows the phone to wake up intermittently as the gsm radio will do (on the measure of microseconds) to see if new data has arrived on the wire. This implementation though, requires its own backend and for each service doing it, we'd be draining the battery even further...

Other work on Android Push:

XMPP/Jabber is even worse. Not only is xmpp/jabber a very chatty protocol, but as it works with one long xml stream, the disconnected nature of a cell phone does not jive well with it... holding an open xmpp connection is probably a very bad idea.

So where does this leave us with push notifications on Android if anything push we do is going to eat the battery and cause the user to uninstall the app?

One option would be to send periodic position updates based on some limiter (the API only has a radius accuracy of no less than a mile. We wouldn't have to wake up all that frequently for the common case of people don't drive around during night time activities). Knowing this, most of the work can be done on the backend to calculate "interesting things," and for notifications, we can send the response over gtalk. Specifically gtalk because most Android phones already have that previously mentioned long-lived tcp session that we can piggy back on. The message could be something along the lines of:

<a href="foursquare://com.playfoursquare.api.NotificationsAuthority/notification/12345?click=user">Mike M.</a> is hanging out at <a href="foursquare://com.playfoursquare.api.NotificationsAuthority/notification/notification/12345?click=venue">Mike's Bar and Grill</a>.

Users clicking on the link would be directed to the android foursquare app, where we could display awesome information.

I'd have to try this out to see if it actually works and how an android app would register as a global URI handler but it should be do-able.

Foursquared Help

2009-07-01T10:07:00.000-07:00

I'm looking for help on foursquared. I can start fleshing out some starter bugs and features so that anyone interested can just jump right in and code.

I've been hating git since I started using it and may end up migrating the SCM to HG on Google Code... I just feel more comfortable with both tools... Still not sure. If I have any external contributors it will really force my hand as I don't want to have to deal with emailed diffs and patches.

I'm not asking for testers because at the pace I'm going, the problems and features I'm trying to implement take up my allocated time plus a bit more. Increasing the water hose of bug reports and feature requests would just drown me. On the other hand, I did put together some crash reporting infrastructure while I was on the airplane so that when I do release, I can get automated bug reports: http://joelapenna.com/blog/labels/dumpcatcher (the code is at http://dumpcatcher.googlecode.com)

Right now I'm working on some troubling network connectivity issues (after some period of usage, all HTTP requests start hanging). After I resolve these issues I will be readed for limited testing... I'm content to release non-feature-complete if only to get some people looking at the app who might be developers.

I'm thinking that feature-wise the release will include the five screens I've sent before: http://joelapenna.com/foursquared/

Fighting LiveFolders

2009-06-30T21:15:00.000-07:00

Cupcake introduced "Live Folders" to the android world and since then I've wondered why the phone does not come with a LiveFolder implementation for bookmarks by default. In an effort to try and get code into core-android I set out to write it, but I've been struggling with all sorts of issues.

ContentProvider.query returns a CursorWrapper, not a real cursor, which means if an existing content provider does not provide a live folders implementation and you want to write your own little content provider to proxy the requests with correctly formatted responses you'll have to implement your own cursor to delegate to the cursor wrapper. I resolved this by creating a MatrixCursor in my content provider into which I copy the columns from the BrowserProvider.

    public Cursor query(Uri uri, String[] projection, String selection, String[] selectionArgs,
            String sortOrder) {
        Cursor c = getContext().getContentResolver().query(Browser.BOOKMARKS_URI,
                Browser.HISTORY_PROJECTION, "bookmark = 1", null, null);

        String[] liveFolderColumnNames = {
                LiveFolders._ID, LiveFolders.NAME, LiveFolders.ICON_PACKAGE,
                LiveFolders.ICON_RESOURCE,
        };
        MatrixCursor mc = new MatrixCursor(liveFolderColumnNames, c.getCount());

        int columnCount = c.getColumnNames().length;
        while (c.moveToNext()) {
            Object[] row = {
                    c.getString(Browser.HISTORY_PROJECTION_ID_INDEX),
                    c.getString(Browser.HISTORY_PROJECTION_TITLE_INDEX),
                    getContext().getPackageName(), R.drawable.ic_launcher_shortcut_browser_bookmark
            };
            mc.addRow(row);
        }
        return mc;
    }

I can't figure out how in hell I'm supposed to use ICON_BITMAP to set an icon for each item in the LiveFolder -- Looking at the source code it looks like its expecting a byte[], which I am in fact providing, but then the provider is having difficult constructing a bitmap out of that data and NPE's trying to (and I don't know why it is doing so) thumbnail the icon!

Dumpcatcher: Uncaught Exceptions and Client IDs

2009-06-29T09:09:00.001-07:00

Alright, about halfway through my flight and I now have an Uncaught exception handler that will post messages to dumpcatcher.

It doesn't work with android at the momement... There is a thread I know I have to read on android-developers. I think I starred it because I knew I'd find it uuseful.

The first user of this class was going to be Foursquared, but it turns out the code I wrote doesn't work.

I quickly added the ability to create a client id attached to each crash in order to allow me track each individual client and in the process refactored a bit of the code here and there. I'll have to remeber to update the design doc as I have made some changes to the design after noticing flaws in my original work.

Dumpcatcher: Java Client

2009-06-28T20:40:00.000-07:00

I'm on my way back from Cambodia... More on that later...

After spending about two-three hours working on hmac signing with Java I now have a working Dumpcatcher client for Java, this is similar to the python client

clients/python/logging/handler.py:Dumpcatcher

in that its usage is:

Dumpcatcher dc = new Dumpcatcher(PRODUCT_KEY, SECRET, "http://localhost:8080/add", 2);
HttpResponse response = dc.sendCrash(
    new NameValuePair("short", "Some short dump"),
    new NameValuePair("long", "some long dump")
)
    assertNotNull(response);
    assertEquals(200, response.getStatusLine().getStatusCode());

Have a look: http://code.google.com/p/dumpcatcher/source/browse/clients/java/src/com/googlecode/dumpcatcher/logging/Dumpcatcher.java?r=20090629r0

Now onto making a logging client for this guy.

Dumpcatcher: Python Client

2009-06-22T21:25:00.003-07:00

For the python dumpcatcher client I decided to go with the standard python logging infrastructure. While there was definitely something lacking in the python documentation for logging.py I found all the information I needed with a combination of the docs and the source code itself. Creating a python client for the dumpcatcher is now pretty simple:

import handler  # from dumpcatcher.clients.python.logging

logger = logging.getLogger('my_tag')
logger.setLevel(logging.DEBUG)  
logger.addHandler(
      DumpcatcherHandler('agtkdW1wY2F0Y2hlcnINCxIHUHJvZHVjdBgCDA',
                         '9b6c65910428419db0d0b730278b72e3',
                         'http://localhost:8080/add',
                         5))


try:
  'a' + 1  # TypeError!
except Exception, e:
  logger.exception('Oh man! Look at this!')

The output of which looks something like:

2009-06-21 06:36:57

demo

handler.py:110:broken Oh man! Look at this!

handler.py:110:broken Oh man! Look at this!
Traceback (most recent call last):
File "/home/jlapenna/code/dumpcatcher/clients/python/logging/handler.py", line 108, in broken
  'a' + 1
TypeError: cannot concatenate 'str' and 'int' objects
File "/home/jlapenna/code/dumpcatcher/clients/python/logging/handler.py", line 110, in broken
  logger.exception('Oh man! Look at this!')

Looking at this now, it seems that all I have left is the java client and data aggregation before I'm feature complete!

Dumpcatcher: Python HTTP and Dates libraries

2009-06-22T21:24:00.000-07:00

Sheesh,

Python really shows its age when you have to deal with dates, http and urls. I was able to implement my signing and verification bit of code just now and it was quite a pain. I have a sample client in clients/python/example.py and the endpoint/crash.Add handler should show you the inverse side of the handshake.

The thing that was definitely most challenging was writing the example client. Because the python libraries for http, etc evolved over time there are about seventy-five ways to do an http request instead of python's normal "one."

Same can be said for datetime modules. Long ago when we had to use mx time there was a DateTime object, now its built into python so everything should be peachy? Yeah, date handling in python was improved by this change but timezones are still outrageous! Considering I'm now about 10 hours off UTC, I have had to think about how to submit a request in string form. For now though I'm going to punt and require that times submitted are in the UTC time zone.

Now that I've got that baked in, I can acutally write the code that logs the requests to the data store. After that, I need to write a bit of code to do sorting and aggregating of the crash dump data.

Dumpcatcher: HTML

2009-06-22T21:22:00.004-07:00

Wow, its been quite a while since I've written any HTML. Probably about one year. Its also been that long since I last used AppEngine and as a result I'm really, really rusty.

I'm now about 3 hours into the hack-a-thon (on a plane instead of of a coffee shop) and I have about 10 hours of battery life remaining.

Things I've learned so far:

I totally forgot the basic IO operations for the app engine data store and with this kind of knowledge its actually pretty hard to re-learn -- I remember such random bits and pieces I forget which parts are bad memory or actuality.

The design doc was fun to write, I had to think about a few different problems that I figured I would encounter. I even realized that my initial idea for request signing had some security vunerabilities which I had to think about how to resolve.

I have one third of the whole project done, but its the smallest one third. Users are now able to register their account and create an unlimited number of products.

Next, I need to write the server side component of crash logging. Eg, when a user submits a crash the request must be authorizedand then it needs to get into the datastore.

Dumpcatcher: Design Doc

2009-06-22T21:22:00.000-07:00

As I do with any project at work, I want to put together a short doc describing the scope and scale at which I will write this app.

Summary

Dumpcatcher is a simple web service that takes authorized requests from remote clients and logs key-value pairs in its datastore for future analysis. These pairs are typically an aribitrary identifier and an exception/stack-trace.

Features

Crash Stack Message storage

Clients must be able to submit stack traces (well, arbitrary strings) along with various bits of meta-data. Version, app name, etc.

Client libraries for Python, Java

I am targeting my <a href="http://joelapenna.com/git/foursquared.git">Foursquared Android Client
Unknown end tag for </a>
as well as any other pet projects I may use in the future.

Data Aggregation

I plan on allowing data aggregation by exception type, custom label and line.

Authenticated Client Requests

All requests by clients must be sent by authorized clients to prevent the service from becoming a black hole for spam. Design:

Design

App Engine has a very simple data store and webapp framework that I intend to utiltize for the basic functionality of the app.

Users

Users represent a single Google Id and a particular developer using the system.

Products

A product is an application that uses the dumpcatcher to log crashes. A user may have multiple products.

Each product registered will have two values associated with it, a productKey which will be passed as a paramter in all HTTP requests to the server and a secret which will be used to HMAC sign a request.

Product secrets will be randomly generated UUIDs.

HTTP Request

All requests to the dumpcatcher service will be secured with an HMAC hash. The hash will be keyed by a unique identifier provided to the client

productKey

Each client -> server request will include a productKey, an identifier used to differentiate between different products using the service.

HMAC

All requests must be submitted with an HMAC-SHA1 hex digest of the request query paramters as well as an increasing "request" identifer. The message consists of a standard http "query", sorted by keyname and quoted, request

For: http://localhost/add?product_keyd=1234&some=pair&other=pair we would construct the digest like so:

TODO(jlapenna): Probably don't want to split on & if the contents of the request might contain one though, they should already be encoded. Something like that...

sorted_query = ''.join(sorted(request.query_string.split('&'))) hash = hmac.new('SOME KEY', sorted_query, hashlib.SHA1)

And, as such, the actual request made to the server will be:

'http://localhost/add?product_key=12345&some=pair&other=pair&hmac=%(hash)s'

On the backend the server will take the reverse steps and using the secret associated with the provided productKey, will verify the authenticity of the request by encoding the query paramters the same way it is done on the client, keying the result by secret associated with the provided productKey.

Datastore

Initially there will be three models, one corresponding to "crashes," another to "users" and the third to "products."

Each user will be associated with a specific Google ID but a single Google ID can have many products.

Security

Security and validity of client-> server requests will be handled via the usage of HTTPS for securing communications and for HMAC to verify authenticity of a client request.

Replay Attack

An attacker with access to the HTTP stream a client -> server request is sent over will be able to execute a replay attack by capturing the HTTP post made by the client and submitting it as its own, at any rate he so desires.

The solution as such is to only allow requests over HTTPS. This gains the added advantage of preventing any private data from leaking via a network observer packet sniffing.

Caveats

It is likely and highly reasonable that an app like this exists in a highly more polished and featureful way. I chose this project because I felt like it would be a good way to explore some new technologies and have a fun time; not because this is in any way "new" or "exciting"

Dumpcatcher

2009-06-19T11:06:00.000-07:00

I took the day off in order to spend it writing some code. My goal for today is to finish with a fully-functional exception catcher for my projects so that I don't have to require users to send me tracebacks or exceptions when they occur.

I started by registering http://dumpcatcher.appspot.com. I will be pushing frequently to this site as I add features. I chose App Engine because I like that its hassle-free application deployment. Launch the app, run it, walk away and it should just work!

Second, I registered http://dumpcatcher.googlecode.com. I will be pushing the code here pretty much just as frequently as I push the site. I chose Mercurial as the SCM because I am sick and tired of git.

So... Let me begin...

Addendum: I actually ended up going into work instead of taking the day off and coded this on a flight to Cambodia.

Foursquared!

2009-06-15T23:25:00.000-07:00

In early April I started working on an Android client for Foursquare I started out with an idea that I could screenscrape enough of the mobile app to make a worthy native api but I was quick to discover other nefarious methods to get at the API that Foursquare was using on the iPhone client. Armed with wireshark and a couple of python scripts, I was able to auto-generate a Java API based on data samples pulled from the TCP dump.

Over the next several weeks I started designing a UI for the app. First and foremost I developed a venue search + checkin activity. I then implmentented another screen to scratch an itch, the CheckinsActivty. This is neat because it displays in a list and in a map which each of your foursquare friends may be.

I had to take a break from the UI work when I started talking with the foursquare people intesting their public API. After several days of fighting with Apache's HttpClient and oauth-signpost (Signpost, by the way, is awesome -- The problems were all in my implementation).

Now its mid-June and I've gotten back to working on UI features; I wrote a "UserActivity" yesterday that displays information about a user. At the moment this information is limited to their name, city and badges acquired. I've also started recovering (fixing bugs) from a lot of refactoring I've been doing in the UI to simplify the design.

I have to start work on figuring out what I want the app to do... In the short term and the medium term. The end state is pretty clear -- feature parity with the iPhone client and a pretty UI to boot. After that there are cool pie in the sky ideas but I'll probably be bored of working on the client by the time I get there.

Join the Foursquared Mailing List!

Twitter blah blah blah

2009-05-24T15:54:00.000-07:00

This weekend I was in Amsterdam and one of the cool things about this trip opposed to last year's tour-de-Europe was that I was checking twitter along the way. I'd stop at a cafe for a beer or a snack and pull out my phone. I'd reload the search.twitter.com page I had in the browser with q=amsterdam and I could get an up to the minute feed of what was going on in the city *right then and there*. I was also able to solicit suggestions from other people using the service and found a set of cool and not so cool things to do in the city! The coolest was that I found out the World Photo Press Exhibition was going on (incidentally in the old church within the red-light district).

I can see where this is going and hopefully in the future I'll be able to filter out the boring "Look at me, I'm in insert-location-here" tweets and get a more useful result set but even now it was quite fun looking to see what was going on around me that was just out of sight.

Coming Soon!

2009-04-23T02:02:00.000-07:00

Lately

2009-04-05T10:11:00.000-07:00

Lately I've been:

Playing Kickball
Hacking on Android/phone stuff
Building cellular networks
Slacking at work
Working at work
Drinking a bit too much, then just the right amount.
Walking around
Eating BBQ while watching a basketball game
Exploritorium'ing the Outside
Planning
Scheming
Reading: 2 books a month (only once was I behind schedule!)

Things I haven't been doing:

Taxes
Selling my car
Keeping in contact
Running around

Sweet Icon, TC!

2009-03-26T18:27:00.001-07:00