Talk:Implementations

Atom feed 30W-compliance

I just added the atom feed and it seems to be 30W-compliant

After some thought, I suppose it would be somewhat compliant. However, in some cases it's too compliant, or compliant in the wrong ways. After 00:00 local time, it automatically switches to the next day, but keeps the old Dow value and rehashes. East of 30W that behavior is desired, since you want to use the previous day's Dow anyway. West of 30W, however, it means we get an "intermediate" hash that's invalid for us. This much has always been a problem with the implementation. Then, sometime after 09:30 ET (I don't know precisely when), it detects the new Dow value and updates the hash... but it does it both west of 30W where it is needed, and east of 30W where it is undesired. In that case, you would get some other "intermediate" using the current Dow with the current day, which isn't what you want. That's a new problem the 30W Rule created.

In summary, it seems to produce invalid hashes between 00:00 (local) and ~09:30 EDT (UTC-4) west of 30W, and between ~09:30 EDT (UTC-4) and 24:00 (local) east of 30W. That is, it's invalid from 00:00-~06:30 in US Pacific Daylight Time (UTC-7), from ~14:30-24:00 British Summer Time (UTC+1) or from ~21:30-24:00 Australia Western Standard Time (UTC+8). I'm not even sure I know what would happen with the feed in New Zealand, where 09:30 USEDT = 01:30 NZST the next day. It'll be even worse come late October/early November, when Australia/NZ switch into DSTand the US switch out... the gap will become 2 hours wider over a couple weeks' time. And of course, these problems are mitigated over weekends, when you have several identical Dow values in a row. Someone should check this out before the weekend, as I'll be busy until Friday 19:00 UTC. If anyone does, and has thoughts, hit me up on my talk page.

Sincerely, your friendly local (global?) 30W-compliance "expert"... --Tim P 15:34, 29 May 2008 (UTC)

I have just updated the feed so it should be 30W compliant. Let me know if it's still a problem. --Xxv 18:03, 30 May 2008 (UTC)

xkcd.com/geohashing Problems?

Hey - I can't get to [1], and none of the graticules' maps are working - is this just me, or the rest of the world, too?

Yes, irc.peeron.com is currently down (as of 20080523 11pm PDT), I know, I'm trying to get hold of someone who can do something about it. Zigdon 07:29, 23 May 2008 (UTC)

Why is the the xkcd.com/geohashing redirecting to irc.peeron.com anyway, shouldn't it be moved to xkcd? I think too many active graticulees are melting the server. --Opspin 08:07, 23 May 2008 (UTC)

it's working again!

Difference between the comic and the generator

Zigdon, xkcd:

The hash in the comic for 2005-05-26-10458.68 starts with db9318........

Here's debugging info for the map page's calculation of http://irc.peeron.com/xkcd/map/:

Graticule: (37, -123) - (38, -122)
Market open on 2005-05-26 = 10458.68
MD5(2005-05-26-10458.68 ): 357e5cac889681628fdd754c1a235919
Split: 357e5cac88968162, 8fdd754c1a235919
offset = 0.20895938122029104, 0.5619729338451526
37.20895938122029 -122.56197293384515

On my machine: $ md5 -s "2005-05-26-10458.68"
MD5 ("2005-05-26-10458.68") = db9318c2259923d08b672cb305440f97

Any idea what's going on here? Is that an extra space on the end there, Zig?
--FunkyTuba

   I've found a bunch of opening values at http://irc.peeron.com/xkcd/map/data/
   with carriage returns on the end, which I dont think were being stripped out before
   hashing. The website went down before I could fully confirm though.
   --ZorMonkey 11:49, 23 May 2008 (UTC)

That was indeed the case, fixed now. Zigdon 14:07, 23 May 2008 (UTC)

Python-CGI-JSON-o-Rama implementation

Hey so i've made a python version of the reference implementation that returns a JSON parseable object containing the lat/long when given a date, xloc and yloc. It uses the XKCD cache of dow values. I'll put it online in the next 5 mins :D

Here it is, fairly simple actually, hope someone finds it useful :)

Update: I've tried to make it classify the region recommended for a particular day using the euclidian distance of the colour of the image provided by Google Maps

The code is on the link above, tell me what you think :)

SinJax

Cool! I was thinking of making it act as a web service, but for reference implementation it's more important to be simple than useful :) Zigdon 14:53, 21 May 2008 (UTC)

I mostly canabolised yours anyways! :) But yeh here you go. The webservice version returning AJAX from python can be found: - AJAX geohashing - Example using the web service

The implementation also has the ability to tell you whether the particular area is water, park/forest or other. It does this by checking the colour on google maps and doing a thresholded euclidian distance check against the expected colours for those things. Also If its water it tells you its probably not accessible I wanna try to add more features like this, things like rating locations based on food and letting you say how far your willing to travel

Thanks to User:Sigmund Fraud from the IRC channel for the space on his server with python installed - SinJax

I've added my own python implementation to the main page (only 5 lines) -- lilac

_very_ nice, im in awe! Though it must be said my concentration was usability as a web service and the implementation of that imaging stuff. Still. Very nice stuff. I love python :) - SinJax

Added a shell script implementation to the main page (slightly shorter than the Python version; let the flames begin) -- gnomon

deeply awesome; no flames from me -- lilac

${var:x:y} slicing isn't portable, so you should really call it a "bash script" and change the #!/bin/sh. I came up with nearly the same thing:

(echo 16i; echo -n "$(date +%F)-$(GET xrl.us/djiopen | col)" | md5 | sed 's/.\{16\}/0.&p/g' | tr a-f A-F) | dc

On Linux, it's md5sum, which prints a "-" for stdin, but dc happily eats that. Your web interface for getting the Dow data is much better/more flexible than that ganky little shortcut url to Y! Finance (they have historical data in .csv too, but at a different interface so I didn't include it), but consider using sed instead. You only have to fire up one dc at the end of the pipeline which is much cleaner IMO.

To make it print out my full co-ordinates, I created a graticule file:

echo -e '42\n-71' > graticule

and appended

| paste -d'\0' graticule -

to the pipeline. If you wanted it put it in a script file as you have and pass them as args, you could (untested):

| while read f; do echo $1$f; shift; done

I'll stop here before I get any farther offtopic :) --Decklin

Error on page?

Just zooming in and out today gives me random meet up locations, any one else having this issue?

Yes, I am experiencing this as well.

It seems to change everytime the applet is reloaded (refreshed, updated, zoomed, or moved) Is there any one who can fix this?

I'm not seeing this? What browsers are you using? Zigdon 17:48, 23 May 2008 (UTC)

I'm experiencing this problem on IE6. MOE37x3 18:10, 23 May 2008 (UTC)

I get the error in IE7 but not Firefox.--Ahecht 18:57, 23 May 2008 (UTC)

Same here, except I us IE6, not 7. Firefox fixed my woes, though. - MythGuy

I, too, am experiencing this. I used the the alternative locater though. http://geohashing.electronicwar.net/

I hope we can we can get this fixed soon. - MythGuy

With the debugger on, I noticed that "00" is appended to the DOW opening every time I click "Update." This changes the hash each time, creating random locations. Zigdon, we need your help. 16:33, 23 May 2008 (UTC)(James)

Ah. Well this is creating problems for me. I used the alternate locator, got location A, went to my local page to see if others might be planning on coming, saw a different GPS (thus giving me location B), I went to the real map and found that the first location that it pops up with is always the same, so I have three locations now... What do I do? - MythGuy

Gah. Ok, I can reproduce this in IE6. I'll see if I can find some simple hack, but doing browser specific code is evil. Zigdon 21:36, 23 May 2008 (UTC)

Ok, it should be fixed in the (beta) version: http://irc.peeron.com/xkcd/map/map2.html. This will become live once the 30W decision is final. Zigdon 21:47, 23 May 2008 (UTC)

bug in reference implementation

The reference implementation is biased against results in the .0..., .0... range. This seems to be because the code that converts a hash string to a fraction stops dividing as soon as the remaining characters are all false (zeros.) This division should probably happen a fixed number of times, instead. 76.121.107.210 17:26, 21 May 2008 (UTC)

Good catch, fixed Zigdon 18:18, 21 May 2008 (UTC)

Great! I also noticed another issue with the perl script -- the page you're retrieving the DJIA value from does not currently have the value for today, so the script is accidentally using the value for yesterday instead. this page is not currently showing a value for 5/21, which the script expects to be there. 76.121.107.210 20:04, 21 May 2008 (UTC)

True - the cron job on the server doesn't get the data from that page, so it's not an issue there. For the sample.pl, perhaps I'll edit it to use a different page if you're trying to load the coords for today. -- Zig. 159.153.4.51 05:15, 22 May 2008 (UTC)

The comic is also a bit vague about exactly how the hex value is converted to a decimal value. One would assume from the description that the hex value is simply converted straight to decimal (FF -> 255) and then appended to the lat/lon coordinates; the reference implementation, however, shows that the hex number is instead converted as a decimal fraction (0.FF -> 0.99609375). This latter approach inherits all of the accuracy problems inherent in converting between decimal and hex/binary fractions. Can't do much about that now except whine, I guess. Ooh, and a cheese platter? Mine's the chunk of reggiano parmesan, thanks. -- gnomon

Yes, the comic states you take the hex as a fraction. Not 'abcdef' but '0.abcdef'. What accuracy problems? you can describe any number between 0 and 1 to the nearest 1/2**16. That translates to about 5 feet or so? Or am I missing something? Zigdon 23:54, 22 May 2008 (UTC)

Actually, that would be 1/(16^16) for 16 fractional hex digits, which translates to something on a microscopic scale. --Tim P 22:18, 23 May 2008 (UTC)

"I'm sorry, our measurements show you missed the meetup by one ten-thousandth of an angstrom."

"Only because I was moved when you measured me!"

--Xkcd 16:36, 24 May 2008 (UTC)

Cookie/Bookmark?

Is it possible to set one's browser to always start at a certain point when viewing the geohash calculator? The first time I visited it (when I was at home in Berkeley), it started out zoomed to my home graticule. Now that I'm at my parents' (in Los Angeles), it defaults to the Boston graticule. I don't know if this is something to do with our respective internet providers or if something has changed with the mapping program, but it would be great if there were an option to default the map to the current view (whatever that may be). If there's already something that I'm missing, someone can just enlighten me. Thanks!!! Darcy 02:15, 26 May 2008 (UTC)

Yes, that would be useful. Done. Zigdon 06:56, 26 May 2008 (UTC)

Awesome, thank you! Darcy 16:24, 26 May 2008 (UTC)

Atom Feed

I noticed yesterday that before the stock exchange data becomes available, the atom feed creates a kind of 'intemediary' geohash using todays date and yesterday's data. This might be intentional, but personally I find that undesirable on weekdays. Could it be modified to show yesterday's date and geohash until data is available? Nicktaylor 14:04, 23 May 2008 (UTC)

I'm currently working on getting an alternative Atom Feed implementation ready for public use (although I think I'm going to wait until the Time Zone issue is resolved). Mine will only update at 09:40 Eastern daily (accounting for DST changes), and uses the irc.peeron.com source for the DJIA, which is updated at 09:35.

I will certainly post it here when it's done, so stay tuned. --Tim P 20:10, 23 May 2008 (UTC)

The time zone issue has been resolved with the 30W Time Zone Rule. I'll continue to work on getting my implementation ready. --Tim P 04:34, 24 May 2008 (UTC)

I don't know if I'll bother with my Atom implementation. I copied your original request at Implementations; I'll keep an eye on it there. --Tim P 05:17, 26 May 2008 (UTC)

Missing Historical Data

There is a good chance that no one cares, but the reference Geohashing Coordinate Calculator will not work for several valid dates. I think some just havent been copied from the previous day, but some may have just been missed. I've put a list of those dates up, delete it if you'd like. --ZorMonkey 14:20, 24 May 2008 (UTC)

FunkyTuba regenerated the data, using a more complete source, so there shouldn't be any more days missing. Do let me know if you find more. Zigdon 08:34, 26 May 2008 (UTC)

Is it just me, or is the shell script borked?

Shell script bombs out for me in two places :(

This:

[[ "$DOW" != +([0-9.]) ]] 

Was always returning true, so exiting the script with a "DOW not available yet.". I replaced that with

[[ -z "$DOW" ]]

to get past it.

then it falls over elsewhere with:

./geo.sh:1: unrecognized modifier `0'
./geo.sh:1: unrecognized modifier `1'
,

For reference, the script reads:

#!/bin/zsh
DOW=$(wget -qO- http://irc.peeron.com/xkcd/map/data/$(date +%Y/%m/%d))
[[ "$DOW" != +([0-9.]) ]] && echo "DOW not available yet." && exit 1
MD5=$(echo -n "$(date -dW30 +%Y-%m-%d)-$DOW"|md5sum|cut -d' ' -f 1|tr a-f A-F)
echo "$1$(echo 10k16i0.${MD5:0:16}p|dc), $2$(echo 10k16i0.${MD5:16}p|dc)"

does it work for other people?

And by the way, is the date modifying in the right place? Shouldn't it retrieve yesterday's dow and caculate with today's date in 30W cases - rather than retrieving today's dow and using yesterday's date for the hash?

I've so far worked out that it's failing on the line:

echo "$1$(echo 10k16i0.${MD5:0:16}p|dc), $2$(echo 10k16i0.${MD5:16}p|dc)"

Better yet, I've further narrowed it further. The parts my system doesn't like are:

echo "$1$(echo 10k16i0.${MD5:0:16}p|dc), $2$(echo 10k16i0.${MD5:16}p|dc)"
                             ^ ^^   ^^                          ^^   ^^  

It doesn't understand the numbers and it reports "command not found" for "dc"

--Psud 08:49, 26 May 2008 (UTC)

I've rewritten the script to incorporate some earlier suggestions I made (they were left on the main page's talk page when the implementations were moved) about portability (you seem to have been bitten by the slicing issue I was worried about). However, you will need to have dc installed no matter what. --Decklin

Shell script implementation

While it runs on sh, ksh, and bash now, it still doesn't work on OpenBSD. If GNU date (which has the -d option) is not detected, I assume FreeBSD's -v is available; OpenBSD doesn't appear to have a comparable feature. Could someone test on OS X? No idea what's there (pretty sure they ship with wget, though).

Super brownie points if you can make it fallback to wget *or* fetch in one line so I can delete that stupid usage warning and have it run on FreeBSD out of the box. --Decklin

Could the user at 63.201.144.114 explain what broke? No math is being done in the shell in any case, and nothing added requires the script to be run under GNU bash instead of any POSIX sh. Extra features are nice, if they can only be implemented in the script itself, but that's an entirely separate issue.

iPhone Implementation

Last night I finished a rough implementation of this to the iPhone 2.0 Beta SDK. Right now it uses your current location and date and then does the math, then gives you a map with the destination address along with directions to there from where you are at. This will be even more useful if the next 3G iPhone come with GPS, until then it uses it's less accurate cellular and wifi location tools to locate you. I plan on adding a way to select your date and your desired graticule as well. Eventually I'll also investigate filtering of the results in watery areas. I can't distribute it currently, but as soon as I can I'll make it available. -- Shakedown

Mapping tool appears borken (5/25/2008)

My test case:

• Go to Mapping tool.
• Zoom out a few, to see the US West coast.
• Center-up on Santa Cruz (if you don't know where that is, use San Francisco.)
• Zoom in a few, to get a nice view.
• Press UPDATE button. (Btw, I'm on Safari, on Mac OS-X/Leopard-latest.)

EXPECTED: A map-pin and coordinates that you can review. ACTUAL: Pin & coordinates flash for a brief moment, then map recenters on Boston, with information gone.

NOTE: Things all worked fine yesterday and the day before (5/23-24.)

Aside: Is there a "bug reporting system", or is this it?  :)

Thanks! Ted 18:55, 25 May 2008 (UTC)

I can't seem to reproduce this in FF or Safari on OSX 10.4.11. Does safari have a javascript error console where it might be logging the error? Zigdon 19:10, 25 May 2008 (UTC)

As far as bug reporting, this is one way, a better was is to come into IRC and talk to me there. Go to http://mibbit.com, select "foonetic" from the pulldown server, and #geohashing for the channel. Zigdon 19:10, 25 May 2008 (UTC)

The Perl reference implementation

I've been playing with the Perl reference implementation, and as I'm in Australia, I found it annoying to deal with its poor 30W capability. So I "fixed" it. Perhaps.

<snip> I updated the sample.pl to have a '-e' flag. If you run it with it, it will report the values according to the 30W rule:

$ ./sample.pl 2/1/08
Downloading DJIA from google: http://finance.google.com/finance/historical?cid=983582&startdate=Jan+25,+2008&enddate=Feb+01,+2008
DJIA opening for 2008-02-01 is 12638.17
Date: 2008-02-01, DJIA: 12638.17
MD5(2008-02-01-12638.17): 917a4419e1345e42569bad9fa5ef5a17
Split: 917a4419e1345e42, 569bad9fa5ef5a17
Fractions: 0.5682718814070582, 0.3383129610792229

$ ./sample.pl -e 2/1/08
Adjusting for 30W
Downloading DJIA from google: http://finance.google.com/finance/historical?cid=983582&startdate=Jan+24,+2008&enddate=Jan+31,+2008
DJIA opening for 2008-01-31 is 12438.28
Date: 2008-02-01, DJIA: 12438.28
MD5(2008-02-01-12438.28): ec6464f6d4d0da51c0c8fe874be8601c
Split: ec6464f6d4d0da51, c0c8fe874be8601c
Fractions: 0.9234068968406207, 0.7530669288934475

Zigdon 21:36, 26 May 2008 (UTC)

Thanks --Psud 10:58, 27 May 2008 (UTC)

Wrong Market Data for 2008-05-30

The official reference implementation has the wrong market data for Friday May 30.

Debugging gives: Market open on 2008-05-30 = 12593.87 While finances.google has 12647.36 --80.216.129.239 14:12, 30 May 2008 (UTC)

• Can we assume it's getting the Dow Jones info from a feed, and thus the feed is wrong? Djomp 15:02, 30 May 2008 (UTC)
  • My bet is that wherever they got their info from is correct, but it wasn't yet updated for today when the webpage hit it. --161.49.253.254 15:21, 30 May 2008 (UTC)
  • The biggest problem to all this is that it means the Saturday meetup locations are wrong, too. Darcy
    • Perhaps the peeron service should hit up Google every five minutes from, say, 09:35 to 10:00, just in case? I've also noticed that this server's time is ~6-7 minutes fast, so a 09:35 hit could have actually been executed at 09:28, before the market opened... --Tim P 15:25, 30 May 2008 (UTC)
    • Even better, if the first hit returns the exact same number as the previous day, keep trying until something changes or it gets to be, like, noon... --Tim P 15:46, 30 May 2008 (UTC)

• Direct future discussion on this topic to Template talk:Expeditions/2008-05-31.
  • This has been corrected. --Tim P 19:12, 30 May 2008 (UTC)

Wrong market data for June 2008 in locations east of -30°

The official reference implementation is attempting to fetch the wrong market data for dates in June 2008 for locations east of -30°.

• For 2008-06-01, it attempts to fetch market data for 2008-06-30.
• For 2008-06-02, it attempts to fetch market data for 2008-07-01.
• For 2008-06-29, it attempts to fetch market data for 2008-07-28.
• For 2008-06-30, it attempts to fetch market data for 2008-07-29.
• For 2008-07-01, it attempts to fetch market data for 2008-06-30 (correct).

Locations west of -30°: tested 2008-06-01, which fetches correct data. Not sure of the extent of the issue, more testing needed. - r0d3n7z 18:07, 30 May 2008 (UTC)

Further testing reveals that the problem is not confined to June 2008, but affects all months with less than 31 days for dates after the 30W rule was applied. Examples:

Date	Market data fetched
2008-06-15	2008-07-14 (wrong)
2008-07-15	2008-07-14 (correct)
2008-08-15	2008-08-14 (correct)
2008-09-15	2008-10-14 (wrong)
2008-10-15	2008-10-14 (correct)
2008-11-15	2008-12-14 (wrong)
2008-12-15	2008-12-14 (correct)
2009-01-15	2009-01-14 (correct)
2009-02-15	2009-03-14 (wrong)
2009-03-15	2009-03-14 (correct)
2009-04-15	2009-05-14 (wrong)
2009-05-15	2009-05-14 (correct)
2009-06-15	2009-07-14 (wrong)

Hope this helps pinpoint the problem. -- r0d3n7z 18:32, 30 May 2008 (UTC)