Difference between revisions of "User:Wenslayer/KMLGenerator"

From Geohashing
imported>Wenslayer
(Tidied up page)
imported>Wenslayer
(Updated with perl script replacement, additional features)
Line 5: Line 5:
 
== Solution ==
 
== Solution ==
 
I've submitted a script to the [http://public.perforce.com:8080 Perforce Public Depot]; the script can be downloaded here:
 
I've submitted a script to the [http://public.perforce.com:8080 Perforce Public Depot]; the script can be downloaded here:
* [http://public.perforce.com:8080/@rev1=head@//guest/marc_wensauer/geohash/gen_geohash_kml.sh gen_geohash_kml.sh]
+
* [http://public.perforce.com:8080/@rev1=head@//guest/marc_wensauer/geohash/gen_geohash_kml.pl gen_geohash_kml.pl]
  
 
== Usage ==
 
== Usage ==
 
<pre>
 
<pre>
Usage: gen_geohash_kml.sh -g <graticule> [-s <source file>]
+
Usage: ./gen_geohash_kml.pl -g <graticule> [-i <input file>] [-n <name>] [-l] [-v]
         -g: specify the graticule of interest (form: <latitude,longitude>)
+
         -g: graticule of interest (format: <latitude,longitude>)
         -s: source file to use (file format: YYYY-MM-DD,<DOW>,<LAT>,<LON>)
+
         -i: input file to use (file format: YYYY-MM-DD,<DOW>,<LAT>,<LON>)
 
                 default: '-' for standard input
 
                 default: '-' for standard input
 +
        -n: name of top-level KML folder holding place markers
 +
                default: "Geohash History for {graticule}"
 +
                ("{graticule}" is replaced with actual graticule supplied)
 +
        -l: generate a label for each place marker corresponding to its date
 +
                default is to suppress a label for the place marker
 +
        -v: enable verbose logging
  
 
Notes:
 
Notes:
Line 19: Line 25:
 
* Resulting KML file is displayed to standard out.
 
* Resulting KML file is displayed to standard out.
 
* You may wish to pre-process the source file to only show dates of interest.
 
* You may wish to pre-process the source file to only show dates of interest.
   E.g.: grep "^2009" allcoords.csv | gen_geohash_kml.sh -g 49,-123 > 2009.kml
+
   E.g.: grep "^2009" allcoords.csv | ./gen_geohash_kml.pl -g 49,-123 > 2009.kml
  
 
Reference:
 
Reference:
* http://wiki.xkcd.com/geohashing/
+
* http://wiki.xkcd.com/geohashing</pre>
</pre>
 
  
 
== Examples ==
 
== Examples ==
  
 
Using the source data, Geohash coordinates can be generated for every day since October 1st, 1928 (over 80 years worth), so with a little filtering, you should be able to generate a KML file for whatever date range and graticule you'd like.
 
Using the source data, Geohash coordinates can be generated for every day since October 1st, 1928 (over 80 years worth), so with a little filtering, you should be able to generate a KML file for whatever date range and graticule you'd like.
 +
 +
[[Image:KML Generator - Google Earth screen shot 1.jpg|thumb|right|Victoria Geohash coordinates since 2008-05-21. Note how few Geohash coordinates actually fall in Victoria and the Saanich Peninsula?]]
  
 
Examples:
 
Examples:
Line 33: Line 40:
 
<pre>
 
<pre>
 
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
 
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
       bzip2 -d | gen_geohash_kml.sh -g 49,-122 > 49,-122_all.kml
+
       bzip2 -d | gen_geohash_kml.pl -g 49,-122 > 49,-122_all.kml
 
</pre>
 
</pre>
  
* Generate a KML for the [[Vancouver, British Columbia|Vancouver]] graticule of all Canada Day and Christmas Day dates:
+
* Generate a KML for the [[Vancouver, British Columbia|Vancouver]] graticule of all Canada Day and Christmas Day dates, with each place marker having a label (its date), and modifying the name of the set of place markers:
 
<pre>
 
<pre>
 
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
 
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
 
       bzip2 -d | sed -nre '/-(07-01|12-25)/p' |
 
       bzip2 -d | sed -nre '/-(07-01|12-25)/p' |
       gen_geohash_kml.sh -g 48,-123 > 48,-123_7.1_12.25.kml
+
       gen_geohash_kml.pl -g 48,-123 -l -n "Geohash coords for Canada/Christmas Day in {graticule}" > 48,-123_7.1_12.25.kml
 
</pre>
 
</pre>
  
Line 47: Line 54:
 
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
 
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
 
       bzip2 -d | sed -ne '/^2008-05-21/,$p' |
 
       bzip2 -d | sed -ne '/^2008-05-21/,$p' |
       gen_geohash_kml.sh -g 49,-123 > 49,-123_since_2009-05-21.kml
+
       gen_geohash_kml.pl -g 49,-123 > 49,-123_since_2009-05-21.kml
 
</pre>
 
</pre>
  
:Screen shot:
 
:[[Image:KML Generator - Google Earth screen shot 1.jpg]]
 
 
Note how few Geohash coordinates actually fall in Victoria and the Saanich Peninsula? 
 
  
 
== To Do ==
 
== To Do ==
  
* Convert script from Bash to Perl to run faster.
+
# <s>Convert script from Bash to Perl to run faster.</s>
* Automate retrieval and decompression of source data.
+
#* '''Done''', parsing all the data only takes a few seconds now, as opposed to many minutes.
* <s>Pretty-up each placemark's description to include hyperlinks to this Wiki and peeron map.</s> Done; don't need link to peeron map (you're in a mapping application already!)
+
# <s>Automate retrieval and decompression of source data.</s>
* Add format checking of source data.
+
#* '''Punted'''; I'd rather not, mostly due to [http://www.faqs.org/docs/artu/ch01s06.html#id2877684 The Art of Unix Programming's Rule of Composition: Design programs to be connected with other programs].
* Properly account for W30 graticules.
+
# <s>Pretty-up each placemark's description to include hyperlinks to this Wiki and peeron map.</s>
 +
#* '''Done'''; don't need link to peeron map (you're in a mapping application already!).
 +
# Add format checking of source data.
 +
#* A few regex's, combined with some numeric range testing ought to do the trick.
 +
# <s>Properly account for W30 graticules.</s>
 +
#* '''Punted''', due to #2, this would be left for an exercise to the user.
 +
# <s>Add option to display labels for each place marker.</s>
 +
#* '''Done'''.
 +
# <s>Add option to override default <name> tag for KML document.</s>
 +
#* '''Done'''.
 +
# Add ability to specify more than one graticule at a time.
 +
#* May involve adding support to read from a config file.

Revision as of 05:44, 17 June 2009

Problem

Have you wondered what historical Geohashes have occurred in your graticule? Have you wanted to see them all at once, instead of typing each date in via peeron? Well this script might help you out here.

Solution

I've submitted a script to the Perforce Public Depot; the script can be downloaded here:

Usage

Usage: ./gen_geohash_kml.pl -g <graticule> [-i <input file>] [-n <name>] [-l] [-v]
        -g: graticule of interest (format: <latitude,longitude>)
        -i: input file to use (file format: YYYY-MM-DD,<DOW>,<LAT>,<LON>)
                default: '-' for standard input
        -n: name of top-level KML folder holding place markers
                default: "Geohash History for {graticule}"
                ("{graticule}" is replaced with actual graticule supplied)
        -l: generate a label for each place marker corresponding to its date
                default is to suppress a label for the place marker
        -v: enable verbose logging

Notes:
* You can get *all* retro Geohash data from:
  http://www.amipsychic.net/geohashing.html
* Resulting KML file is displayed to standard out.
* You may wish to pre-process the source file to only show dates of interest.
  E.g.: grep "^2009" allcoords.csv | ./gen_geohash_kml.pl -g 49,-123 > 2009.kml

Reference:
* http://wiki.xkcd.com/geohashing

Examples

Using the source data, Geohash coordinates can be generated for every day since October 1st, 1928 (over 80 years worth), so with a little filtering, you should be able to generate a KML file for whatever date range and graticule you'd like.

Victoria Geohash coordinates since 2008-05-21. Note how few Geohash coordinates actually fall in Victoria and the Saanich Peninsula?

Examples:

  • Generate a KML for the Surrey graticule of all possible Geohash coordinates since October 1st, 1928 (over 29,000 points!):
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
      bzip2 -d | gen_geohash_kml.pl -g 49,-122 > 49,-122_all.kml
  • Generate a KML for the Vancouver graticule of all Canada Day and Christmas Day dates, with each place marker having a label (its date), and modifying the name of the set of place markers:
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
      bzip2 -d | sed -nre '/-(07-01|12-25)/p' |
      gen_geohash_kml.pl -g 48,-123 -l -n "Geohash coords for Canada/Christmas Day in {graticule}" > 48,-123_7.1_12.25.kml
  • Generate a KML for the Victoria graticule of all dates since the original Geohash webcomic (May 21st, 2008):
$ wget -qO http://www.amipsychic.net/coords/allcoords.csv.bz2 |
      bzip2 -d | sed -ne '/^2008-05-21/,$p' |
      gen_geohash_kml.pl -g 49,-123 > 49,-123_since_2009-05-21.kml


To Do

  1. Convert script from Bash to Perl to run faster.
    • Done, parsing all the data only takes a few seconds now, as opposed to many minutes.
  2. Automate retrieval and decompression of source data.
  3. Pretty-up each placemark's description to include hyperlinks to this Wiki and peeron map.
    • Done; don't need link to peeron map (you're in a mapping application already!).
  4. Add format checking of source data.
    • A few regex's, combined with some numeric range testing ought to do the trick.
  5. Properly account for W30 graticules.
    • Punted, due to #2, this would be left for an exercise to the user.
  6. Add option to display labels for each place marker.
    • Done.
  7. Add option to override default <name> tag for KML document.
    • Done.
  8. Add ability to specify more than one graticule at a time.
    • May involve adding support to read from a config file.