Difference between revisions of "User:AperfectBot"

From Geohashing
(Tasks Remaining)
(under new management)
 
(25 intermediate revisions by 5 users not shown)
Line 1: Line 1:
'''While I am moving, [[User:Jiml|Jiml]] is running the bot.'''  I now have internet access again, and will be running the bots again sometime in the near future.  Until then, Jiml will continue to run them. --[[User:Aperfectring|aperfectring]] 17:50, 21 September 2009 (UTC)
+
{{quote||The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.|Eliezer Yudkowsky}}
  
This bot is owned by [[User:Aperfectring|aperfectring]].  It is an implementation of pywikipediabot, and uses some code <s>stolen from</s> graciously donated by [[User:ReletBot|relet]]. Its job is to maintain the future and recent past [[Geo Hashing:Current events|planning pages lists]], and to create new planning pages upon request. Much thanks goes to [[User:Robyn|Robyn]] and others who helped with the planning and suggestions for improvement.
+
This bot was created by [[User:Aperfectring|aperfectring]], who also maintained and hosted it until 2023. Since then, it is hosted by [[User:Fippe|Fippe]].  It is implemented on top of pywikipediabot, and its job is to maintain the daily expeditions lists, as well as the expedition lists on [[Geohashing:Current events]].  It also is able to create both per-user and per-graticule expedition lists.  See [[Help:AperfectBot]] for more information.
 +
 
 +
== Much Thanks ==
 +
* The uses some code <s>stolen from</s> graciously donated by [[User:ReletBot|relet]], and he also provided source control before the move to github. We have started to merge some code which we were sharing, to create a common library of useful geohashing wiki functions.  We hope these functions will help facilitate easier development of new bots, and also improve the overall quality of all bots which use it.
 +
* [[User:Robyn|Robyn]] came up with the original idea, and provided some great input early in its development.
 +
* [[User:Jiml|Jiml]] hosted and ran the bot while I moved across the country and scrambled for internet access.
 +
* All others who helped with the planning and suggestions for improvement.
 +
The bot wouldn't have been completed nearly so quickly, or gotten to be the dumb ape it is without all of you.  Thanks.
  
 
== How it works at the moment I edited this ==
 
== How it works at the moment I edited this ==
Line 10: Line 17:
 
[[User:AperfectBot/Changing_the_output|Go here]] for information about how it constructs the summary.
 
[[User:AperfectBot/Changing_the_output|Go here]] for information about how it constructs the summary.
  
The bot takes about 10 minutes to complete one iteration.  Most of this time is actually fetching and writing the pages, the actual processing time is rather minimal.
+
The bot takes somewhere around 10-15 minutes to complete one iteration.  Most of this time is actually fetching and writing the pages, the actual processing time is rather minimal.
 +
 
 +
== Updates to functionality ==
 +
I'll try to keep this up to date with what changes I make on the Ape, but the best place to look is the github repo, as that will be maintained much more often.
 +
 
 +
=== 2010-04-28 ===
 +
I added dynamic support for holidays, instead of the previous static one, which I never added holidays to.  Holidays included are:
 +
* Geohashing Day
 +
* Mouseover Day
 +
* DJIA holidays (for the day they actually fall on, not when DJIA observes them)
 +
* Easter Sunday (Came for free with DJIA Good Friday holiday)
 +
* Pi Day
 +
* Talk Like A Pirate Day
 +
 
 +
The ape should be able to identify these holiday on its own forever!
  
 
==Tasks Remaining==
 
==Tasks Remaining==
 
These are in rough order of importance
 
These are in rough order of importance
* Update the bot to use the common library for finding users who participated.
+
* Review and possibly improve the fuzzy logic to the library user list function. (ongoing, for continuous improvement)
* Review and possibly improve the fuzzy logic to the library user list function.
+
* Improve transport detection.
* Change the update request page so that it constantly updates old pages.
 
* Add a per-user list of expeditions.  This would be opt-in, and require a format string of some sort.
 
 
* Look into either signing up for access to a server, or convince one of our friendly neighborhood geohashers with a server to let me have access to update and run this, as well as the Notification program, from it.
 
* Look into either signing up for access to a server, or convince one of our friendly neighborhood geohashers with a server to let me have access to update and run this, as well as the Notification program, from it.
 
* Sort the results for each day using an undetermined key to sort on
 
* Sort the results for each day using an undetermined key to sort on
Line 25: Line 44:
 
I will use this section to plan out my time in the evening on tasks.  I will probably put in an hour or two of work on most weekdays.  Anything from before 2009-06-17 is included for historical purposes.  The bot is live!  Anything I will be doing from now on is new features, bug fixes, or improvements to output.
 
I will use this section to plan out my time in the evening on tasks.  I will probably put in an hour or two of work on most weekdays.  Anything from before 2009-06-17 is included for historical purposes.  The bot is live!  Anything I will be doing from now on is new features, bug fixes, or improvements to output.
  
==== 2009-11-03 ====
+
==== 2010-01-27 ====
* Put the bot into source control
+
* The window is now about 10 seconds each run, which is currently lasting about 6 minutes.
* Added a getSectionRegex function
 
* Updated the getSection* functions to be able to operate on just the top level sections or all subsections with an option
 
* Updated the ape to use the common GraticuleDatabase library
 
* Updated the ape to use the getSection* functions from the library
 
 
 
==== 2009-11-02 ====
 
* I did some refactoring on the parsing engine.  It is now a bit more flexible in how it is looking at the expedition page sections, but lost the ability to look into the first section for location descriptions.  None of the pages currently on the [[Geo Hashing:Current events|current events]] page encountered any change because of that change.  I am much more happy about the quality of the new code, and since it is more generic, it should facilitate easier development of new bots or extensions to this bot.  The new code is a generic section/subsection parser.  It is actually two functions, one which will return the body of a section with the name provided, and one which will return all sections and their bodies as a hash where SectionTitle->Body.  What initially prompted me to do this is a request/offer of the user list generator part of the bot for a new bot relet is creating, and I am quite happy with how it turned out.
 
 
 
==== 2009-07-15 ====
 
* Move the help text to [[Help:AperfectBot]] and don't have the bot create the section.
 
 
 
==== 2009-07-13 ====
 
* The summaries have been added to the Main Page and if more than 4 would appear on that page, then a "More summaries..." link is put in their place.
 
* An "Update this list" link has been added to date pages.
 
 
 
==== 2009-07-10 ====
 
* All old expedition summaries have been created.  Date pages have been updated to include the summaries as well.  The next step is to move the update request to someplace more permanent, and provide an edit link.  That will be done after this weekend.
 
 
 
==== 2009-07-05 ====
 
* There appears to be some issue with writing to the old date pages.  Until I can figure out what is causing it, the updates on old date pages will be on hold.  Anything on the current events page should be updated as normal, or with minimal delay.
 
 
 
==== 2009-07-04 ====
 
* I fixed another bug in the list with user lists.
 
* And another where the bot was stripping links from location descriptions.
 
* I also added the ability to specify old dates to update.  I am running a test [[User:AperfectBot/Test_Page|here]] which should update all date pages this year with the appropriate summaries.
 
 
 
==== 2009-07-02 ====
 
* Date pages should now be automatically created for all dates from $TODAY into the future.  They will be overwritten for all dates the current day US ET forward that are on the list.  They will only be written between the hours of 9AM and 10AM US ET.
 
* I fixed some cases where users' names would show up in the user list multiple times.  Let me know if you still see cases where the same user shows up twice.
 
 
 
==== 2009-06-26 ====
 
* Create date pages (YYYY-MM-DD) if they don't already exist.
 
 
 
==== 2009-06-22 ====
 
* Move the output from its [[User:AperfectBot/Test Page2|current location]], to a template: [[Template:Recent_expeditions]]
 
* Change the [[User:AperfectBot/Test Page|old output page]] into a transclusion tag, to preview what the final results on the current events page will look like.
 
* Add includeonly tags around a "provide your own update" link, which goes to an edit page.
 
* Add a "How to change the text of my planning/expedition summary" link, which should be to a simple, user-friendly description of how it chooses the text, and how someone would go about changing it.
 
* Add a "Report a problem" link to the list.
 
* Code fix on writing pages to make them a bit faster.
 
* Status: The content for point 4 [[User:AperfectBot/Changing_the_output|is here]]. Point 1&2 now complete.  Point 5 done, time is now about 8 minutes between updates instead of 12.
 
* Status2: Everything in the list should be done now.  Please look [[User:AperfectBot/Test Page|the output page over]].  Thanks.
 
 
 
==== 2009-06-21 ====
 
* Start using templates and transclusion.
 
** This should (at least in my deranged mind) make dealing with user edits to the daily lists easier to cope with.
 
* Add at least some user modification of the list
 
* Status: The first part seems to be mostly in place, the only part of that which is left is to write the output which is now [[User:AperfectBot/Test Page2|here]] to a template page. I also think I have something ready for test on the second point.  I also added a hook which will allow us to enhance the date links with holidays.
 
 
 
==== 2009-06-20 ====
 
  
* Switching to a new set of categories as follows:
+
==== 2010-01-26 ====
** '''Category:Meetup on YYYY-MM-DD''' for anything from the latest available back to the first in the list
+
* Fixed user texts so that there is a much better chance they will be preserved. The window is still measurable, but much smaller than it used to be.
** '''Category:Expedition planning''' for anything further in the future than the latest available
 
* Still looking for the best way to figure out the last the coords are available for.
 
** My current thought is to use the python implementation posted here.
 
* Status: The above is complete.  The bot also will create empty date stubs now.  I am now looking for input on my update below.
 
  
==== 2009-06-19 ====
+
==== 2010-01-14 ====
* More planning on picking the dates to report.
+
[[User:Aperfectring/Expeditions]] is updating with a new format: <pre>
:* My current thought is to report everything from Expedition planning from three weekdays ago, until the latest available coordinates.  This gives people a bit more time to report on a potentially geohash-busy weekend, but means that the number of days in the recent past list is not constant.  This table assumes no DOW holidays.
 
::{| border="1" cellpadding="5" cellspacing="0"
 
|- bgcolor="lightgrey"
 
!Today (US Eastern Time) !! First day reported !! Last day reported
 
|-
 
|Sunday    || Wednesday || Monday
 
|-
 
|Monday    || Wednesday || Tuesday
 
|-
 
|Tuesday  || Thursday  || Wednesday
 
|-
 
|Wednesday || Friday    || Thursday
 
 
|-
 
|-
|Thursday  || Monday    || Friday
+
|DATE||GRATADD||GRATNAME||PEOPLE||REACHED:[[EXPED|Succeeded]]:[[EXPED|Failed]]:REACHED||LOCATION
|-
+
</pre>
|Friday    || Tuesday  || Monday
+
With the text before and after the update area, this results in a sortable table of expeditions!
|-
 
|Saturday  || Wednesday || Monday
 
|}
 
:* Another option is to have a fixed number of past days in the list (let's say 3), and all days where coordinates are available.  This keeps the recent past list a constant size, but if people are busy geohashing on weekends, their expedition planning could drop off the page before it is reported on.  This table assumes no DOW holidays.
 
::{| border="1" cellpadding="5" cellspacing="0"
 
|- bgcolor="lightgrey"
 
!Today (US Eastern Time) !! First day reported !! Last day reported
 
|-
 
|Sunday    || Thursday  || Monday
 
|-
 
|Monday    || Friday    || Tuesday
 
|-
 
|Tuesday  || Saturday  || Wednesday
 
|-
 
|Wednesday || Sunday    || Thursday
 
|-
 
|Thursday  || Monday    || Friday
 
|-
 
|Friday    || Tuesday  || Monday
 
|-
 
|Saturday  || Wednesday || Monday
 
|}
 
* If I get some decent feedback on which of the above is best, I will begin coding on it.
 
* Figure out how to determine what days there are coordinates available for.
 
* Status: I am now using the first option, and parsing both '''Category:Expedition planning''' and '''Category:Expeditions'''.  It now updates about every 7 minutes with the truncated date list.
 
  
==== 2009-06-18 ====
+
Added some new options:
* Reverse the sort of the dates
+
PEOPLE:x - where x is any number - Will display at most x people in the list.  Otherwise operates exactly like PEOPLE
* Plan out how to pick the dates to report
+
TRANSICON - Will display icons for all transport options detected.
* Status: first point done, second still in progress.
+
REACHICON - Will display either a green (SUCCESS) or red (FAILURE) arrow icon
  
==== 2009-06-17 ====
+
USERTEXT is now preserved across runs.
* Tweak the length of location descriptions
 
* Trim out the extra instances of header boundaries in the location descriptions
 
* Begin work on sectionalizing the results by date
 
* Possibly start the bot in a continuous loop, which means that it will provide updates about every 30 minutes, if needed.  I will leave this going overnight and while I am at work the next day, if I do it.
 
* Status: All of the above complete.  Let me know if the bot misbehaves.  If it starts misbehaving really badly, use the [[User:AperfectBot#Distraction Banana|Distraction Banana]] section below.
 
  
==== 2009-06-16 ====
+
Added ability for multiple users/formats to be specified. See [[User:Relet/Expeditions]] for proof.  Users, expedition list pages, and formats need to be specified here: [[User:AperfectBot/User_expedition_lists]]
* Fix up some location parsing.
 
* Status: Did some work on it, but not a whole lot
 
  
==== 2009-06-15 ====
+
==== 2009-12-02 ====
* Look for more options as far as people going
+
* I have started work on creating per-user expedition lists.  Currently it is making a list for [[User:Aperfectring|Aperfectring]] [[User:Aperfectring/Expeditions|here]].  The entries of this list are currently generated using the following format, but it is likely to change in the future:
* Look for more options as far as the location the hashpoint is in
+
date DATE - gratadd GRATADD - gratname GRATNAME - people PEOPLE - location LOCATION - transport TRANSPORT - reached REACHED:Succeeded:Failed:REACHED - reason REASON - link LINK - exped EXPED - usertext USERTEXT
* Status: The user list may get a little better with time, but its quite close at this pointThere is still work to be done on the location.
+
* Features:
 +
** The reached section is replaced first, so it is able to contain other substitutions seamlessly.  It is used as follows: REACHED:1:2:REACHED  The text in '1' is used on reached coordinates, '2' otherwise.
 +
** USERTEXT will be preserved across updates of the bot, allowing people to make their own comments about their expeditions.
 +
** PEOPLE will contain all of the rest of the people who attended the expedition.
 +
** The use of a format string allows for great flexibility in how the final output is displayed.
 +
** Expeditions are sorted lexicographically by the expedition page name (YYYY-MM-DD LAT LON).  This has the affect of being chronological by date, then pseudo geographical in the following order: Southern hemisphere first, Northern secondThen roughly equator to poles, though there will be some mixing.  After that, it will do Western hemisphere first, Eastern second.  Then roughly prime meridian to "date line".
 +
* Caveats and current implementation holes:
 +
** Hardcoded to only update for Aperfectring.
 +
** Hardcoded to one specific page to update.
 +
** Hardcoded to the format noted above.
 +
** USERTEXT is currently not preserved across updates.
 +
** The page is completely rewritten each time, which prevents people from having non user list text before or after the user list.
  
==== 2009-06-14 ====
+
==== 2009-11-03 ====
* Status: 100% less shouting on the page
+
* Put the bot into source control
 
+
* Added a getSectionRegex function
==== 2009-06-13 ====
+
* Updated the getSection* functions to be able to operate on just the top level sections or all subsections with an option
* More thorough planning
+
* Updated the ape to use the common GraticuleDatabase library
* Begin coding in earnest
+
* Updated the ape to use the getSection* functions from the library
* Status: By the end of the day, I had a very basic parser, which wrote the full contents of '''Category:Expedition planning''' to [[User:AperfectBot/Test_Page| a page on the wiki]].
 
 
 
==== 2009-06-12 ====
 
* Begin preliminary planning
 
  
--[[User:Aperfectring|aperfectring]] 12:05, 17 June 2009 (UTC)
+
==== Older stuff ====
 +
Go [[User:AperfectBot/Change_history|here]] to see older change history.
  
 
= EMERGENCY STOP SECTION =  
 
= EMERGENCY STOP SECTION =  

Latest revision as of 19:03, 19 March 2023

The AI does not hate you, nor does it love you, but you are made out of atoms which it can use for something else.
Eliezer Yudkowsky

This bot was created by aperfectring, who also maintained and hosted it until 2023. Since then, it is hosted by Fippe. It is implemented on top of pywikipediabot, and its job is to maintain the daily expeditions lists, as well as the expedition lists on Geohashing:Current events. It also is able to create both per-user and per-graticule expedition lists. See Help:AperfectBot for more information.

Much Thanks

  • The uses some code stolen from graciously donated by relet, and he also provided source control before the move to github. We have started to merge some code which we were sharing, to create a common library of useful geohashing wiki functions. We hope these functions will help facilitate easier development of new bots, and also improve the overall quality of all bots which use it.
  • Robyn came up with the original idea, and provided some great input early in its development.
  • Jiml hosted and ran the bot while I moved across the country and scrambled for internet access.
  • All others who helped with the planning and suggestions for improvement.

The bot wouldn't have been completed nearly so quickly, or gotten to be the dumb ape it is without all of you. Thanks.

How it works at the moment I edited this

It looks at Category:Meetup on YYYY-MM-DD for the most recent days, and all days where coords are available, and finds all pages in it which have a title that matches: YYYY-MM-DD lat lon

It also looks at Category:Expedition_planning for all pages matching YYYY-MM-DD lat lon where the date is further in the future than the latest available coordinates.

Go here for information about how it constructs the summary.

The bot takes somewhere around 10-15 minutes to complete one iteration. Most of this time is actually fetching and writing the pages, the actual processing time is rather minimal.

Updates to functionality

I'll try to keep this up to date with what changes I make on the Ape, but the best place to look is the github repo, as that will be maintained much more often.

2010-04-28

I added dynamic support for holidays, instead of the previous static one, which I never added holidays to. Holidays included are:

  • Geohashing Day
  • Mouseover Day
  • DJIA holidays (for the day they actually fall on, not when DJIA observes them)
  • Easter Sunday (Came for free with DJIA Good Friday holiday)
  • Pi Day
  • Talk Like A Pirate Day

The ape should be able to identify these holiday on its own forever!

Tasks Remaining

These are in rough order of importance

  • Review and possibly improve the fuzzy logic to the library user list function. (ongoing, for continuous improvement)
  • Improve transport detection.
  • Look into either signing up for access to a server, or convince one of our friendly neighborhood geohashers with a server to let me have access to update and run this, as well as the Notification program, from it.
  • Sort the results for each day using an undetermined key to sort on
  • Let AperfectBot eat bananas

Task scheduling

I will use this section to plan out my time in the evening on tasks. I will probably put in an hour or two of work on most weekdays. Anything from before 2009-06-17 is included for historical purposes. The bot is live! Anything I will be doing from now on is new features, bug fixes, or improvements to output.

2010-01-27

  • The window is now about 10 seconds each run, which is currently lasting about 6 minutes.

2010-01-26

  • Fixed user texts so that there is a much better chance they will be preserved. The window is still measurable, but much smaller than it used to be.

2010-01-14

User:Aperfectring/Expeditions is updating with a new format:

|-
|DATE||GRATADD||GRATNAME||PEOPLE||REACHED:[[EXPED|Succeeded]]:[[EXPED|Failed]]:REACHED||LOCATION

With the text before and after the update area, this results in a sortable table of expeditions!

Added some new options: PEOPLE:x - where x is any number - Will display at most x people in the list. Otherwise operates exactly like PEOPLE TRANSICON - Will display icons for all transport options detected. REACHICON - Will display either a green (SUCCESS) or red (FAILURE) arrow icon

USERTEXT is now preserved across runs.

Added ability for multiple users/formats to be specified. See User:Relet/Expeditions for proof. Users, expedition list pages, and formats need to be specified here: User:AperfectBot/User_expedition_lists

2009-12-02

  • I have started work on creating per-user expedition lists. Currently it is making a list for Aperfectring here. The entries of this list are currently generated using the following format, but it is likely to change in the future:
date DATE - gratadd GRATADD - gratname GRATNAME - people PEOPLE - location LOCATION - transport TRANSPORT - reached REACHED:Succeeded:Failed:REACHED - reason REASON - link LINK - exped EXPED - usertext USERTEXT
  • Features:
    • The reached section is replaced first, so it is able to contain other substitutions seamlessly. It is used as follows: REACHED:1:2:REACHED The text in '1' is used on reached coordinates, '2' otherwise.
    • USERTEXT will be preserved across updates of the bot, allowing people to make their own comments about their expeditions.
    • PEOPLE will contain all of the rest of the people who attended the expedition.
    • The use of a format string allows for great flexibility in how the final output is displayed.
    • Expeditions are sorted lexicographically by the expedition page name (YYYY-MM-DD LAT LON). This has the affect of being chronological by date, then pseudo geographical in the following order: Southern hemisphere first, Northern second. Then roughly equator to poles, though there will be some mixing. After that, it will do Western hemisphere first, Eastern second. Then roughly prime meridian to "date line".
  • Caveats and current implementation holes:
    • Hardcoded to only update for Aperfectring.
    • Hardcoded to one specific page to update.
    • Hardcoded to the format noted above.
    • USERTEXT is currently not preserved across updates.
    • The page is completely rewritten each time, which prevents people from having non user list text before or after the user list.

2009-11-03

  • Put the bot into source control
  • Added a getSectionRegex function
  • Updated the getSection* functions to be able to operate on just the top level sections or all subsections with an option
  • Updated the ape to use the common GraticuleDatabase library
  • Updated the ape to use the getSection* functions from the library

Older stuff

Go here to see older change history.

EMERGENCY STOP SECTION

Putting any text beneath the following header will cause the bot to stop running. Please only do so if the bot is REALLY misbehaving.

Distraction Banana