Author: bakert

Great talk taken from his new book The Stuff of Thought. I won’t cover the sophisticated and interesting arguments here (read the book!) but here are some nuggets on use of language …

Child to her father:

I don’t want the flat crayons I want the ones that look like an audience.

Groucho Marx classics:

I’ve had a great evening. But this wasn’t it.

If I held you any closer I’d be on the other side of you.

Why does a man with hair on his head have more hair than a man with hairs on his head?

Was the World Trade Centre attack on September 11th 2001 one incident or two? There was a 3.5 billion dollar lawsuit based on the interpretation of a maximum payout of $3.5 billion dollars “per incident”.

One of the reasons detectives suspected Scott Peterson of the murder of his wife was that he referred to her in the past tense before her body was found.

James Garfield‘s assassin’s remark following the death of the President of the United States from complications arising from poor medical care after being shot three months earlier:

The doctors killed him. I just shot him.

The Federal Communications Comission chose not to censure Bono over his remark that the award he was receiving was “really, really fucking brilliant” as it did not refer to a sexual act but was rather used to emphasize. Further legislation was passed as a result of this. Although the legislation goes into graphic detail over what words are not allowed it fails to make illegal the use of fucking as an emphasizing adverb.

That “damn tabernacle” and “damn chalice” are swearing in Quebecois French.

The description of a certain mode of speech as “fuck patois”. And an illustrative tirade from a soldier returning from war:

You fucking mean I’ve fucking come all the way fucking home from the fucking war to my fucking house to find my fucking wife having illicit sexual relations with my fucking neighbour?

The editor of Gourmet magazine challenged one of his journalists to bribe the maître d’s of New York’s finest restaurants $50 to seat him and his partner without reservations. It worked each and every time.

Update: these instructions refer to the HAppS, 0.8.8. The latest version (September 2007) if 0.9.1a. It radically alters the HAppS APIs as well as the installation procedure. You can find more information here which is an incomplete tutorial inside one of the new source code repositories.

1. Get the latest version of ghc for your platform:

   http://haskell.org/ghc/

2. Get the latest version of darcs:

   http://wiki.darcs.net/DarcsWiki/CategoryBinaries

3. Throughout, when building HAppS or any of it’s dependencies use:

   $ runhaskell Setup.hs configure
   $ runhaskell Setup.hs build
   $ sudo runhaskell Setup.hs install
   

   and not the makefiles. Even where the README or web page says otherwise. In some instances Setup.lhs will be provided instead of Setup.hs.

4. Use the latest source for HAppS 0.8.8 not the stable version. The stable version won’t build! Get it via darcs not tarballs. The tarballs are broken links!

   darcs get --partial http://happs.org/HAppS
   

5. Try and build HAppS (with the commands in step 3 above), and every time you fail, install the missing packages. The documentation in the HAppS tutorial is up-to-date (as of September 2007) on this score. So you may be better off there than here if it is long after that date.

   The hackage package list is useful but not complete.

   You may need: HaXML, binary, hslogger.

   HList is a special case as it doesn’t have a conveniently downloadable package. Use:

   wget -r -nH -X '*/_darcs' -np http://darcs.haskell.org/HList/
   

   Plus the following are only available via darcs: syb-with-class ( darcs get http://happs.org/HAppS/syb-with-class ), default ( darcs get http://happs.org/HAppS/default ), normalize ( darcs get http://happs.org/HAppS/normalize ), generic-xml ( darcs get http://happs.org/HAppS/generic-xml )

You can test your installation with the “Hello World” in the HAppS Tutorial

Update: the rest of the HAppS tutorial was fairly dense and difficult to understand as well as being somewhat out of date. To remedy this situation I wrote Simple Haskell Web Programming with HAppS.

• 0 Success
• 1 Syntax or usage error
• 2 Protocol incompatibility
• 3 Errors selecting input/output files, dirs
• 4 Requested action not supported: an attempt was made to manipulate 64-bit files on a platform that cannot support them; or an option was specified that is supported by the client and not by the server.
• 5 Error starting client-server protocol
• 10 Error in socket I/O
• 11 Error in file I/O
• 12 Error in rsync protocol data stream
• 13 Errors with program diagnostics
• 14 Error in IPC code
• 20 Received SIGUSR1 or SIGINT
• 21 Some error returned by waitpid()
• 22 Error allocating core memory buffers
• 23 Partial transfer due to error
• 24 Partial transfer due to vanished source files
• 30 Timeout in data send/receive

To designate a particular controller so that all of its methods appear off the root of your site, add the following to routes.rb inside the ActionController::Routing::Routes.draw do |map| block:

    map.connect ':action', :controller => "yourcontrollername"

It’s a bank holiday weekend. What better way to spend it than to try to write a web application in 30 hours?!

I’m going to write an application to search eBay, Amazon, free ebooks, TV guides, cinema listings and down the back of the sofa for books, movies and music. It will offer persistent search so you can say, “I want a copy of Catcher in the Rye but I don’t want to pay more than �3 for it.” It will offer to email you results, or create an RSS feed.

What I’m definitely going to need is a good eBay library and a good Amazon library. Ruby (a language I really like) seems to have both of those. Python has easybay, and some kind of Amazon library but neither seem quite as mature as the Ruby equivalents. I’d like to use web.py with Python as I’ve never really got on with Rails like some people seem to. But perhaps this can be the app that changes all that. So for Hour One at least I’m going to be writing this in Ruby using Rails.

In the spirit of doing “the simplest thing that could possibly work” the initial app is going to be single text input page that searches eBay UK only and displays the results. Under source control, with unit tests, because I don’t want this to come crashing down around my ears 20 hours in.

OK, it’s already 0904 – I’m four minutes behind schedule &#8211 let’s go!

Hour One

We are off! Here’s what I managed in Hour One:

• Created a finder skeleton app.
• Installed darcs for source control and created a repository.
• Spent too long trying to follow the instructions from Rails Recipes to get Rails running without a database, without success.
• Created the Item and Ebay classes with some very basic unit tests.

Now to hook up to the actual eBay library and get that working.

Hour Two

I’ve got the remarkably nice Ruby eBay API client up and running. Now I need to get it to return results to my application.

In more detail:

• Installed subversion, retrieved Ruby eBay API library and then realised that’s for hacking on it not using it.
• Installed Ruby eBay API client gem.
• Set up developer keys for developer.ebay.com
• Created config file for Ruby eBay API client.
• Had some breakfast.
• Generated auth token for eBay sandbox user.
• Created an eBay sandbox user.
• Created test app that gets the official eBay time.
• Created test listings through eBay lib.
• Searched test listings with test app.

Hour Three

I’ve got the eBay client returning results into the application. I need to spend some more time getting attributes like current price and postage cost so that these results are more useful.

I also need to create the web front end for display of the search form and the search results.

• Unit tests for ebay searcher.
• Got ebay searcher returning FinderItems.
• Created another eBay user to bid on my test auctions.
• Had a good poke around the eBay API to see what attributes are available.

Hour Four

I found current price and URL in the eBay API so the FinderItems produced now include those details. I’ve started work on the web side of things. I have a basic search form and basic search results page. Now I just need to get the results page to display the FinderItems I’m passing to it in the controller.

• Identified basic properties in eBay API and added them to FinderItem objects.
• Installed irb for interactive ruby.
• Generated basic rails controller for the app.
• Created basic search form and basic search results page.
• Created layout for header/footer for evey page in the app.
• Started to hook EbaySearcher and FinderItem together with the web view/controller.

Hour Five

Quite a frustrating hour but things are moving on nonetheless. I have eBay search results from the sandbox display in the browser. I’m adding Amazon in before I solidify final design decisions like what attributes FinderItem is going to require so I don’t get caught out reproducing the eBay API with not enough flexibility to cope with other sources of data.

Loads of little tasks are starting to queue up — everything from making the search results into clickable URLs to adding a “please wait” screen. Plus I need to take some time out and do an actual design for the site.

• Spent a little while chasing down a bug – rails doesn’t like you using include in the top-level namespace – you need to do it inside a class else you get “NameError: cannot remove Object”.
• Got eBay results displaying in browser
• Tried to move query => index and ended up deadlocking the webserver and moving it back.
• Installed Ruby/Amazon library. Created AmazonSearcher with unit tests.

Hour Six

I’ve got Amazon and eBay results coming through in the web app now. There’s some serious issues with the eBay side of things (tries to retrieve more than 10,000 items from the API if you enter “test” as the search term) but looking good. I’ve also hidden the two existing data sources behind a Searcher class that keeps knowledge of exactly what’s going on away from things that shouldn’t know about it. Didn’t get much else done this hour – was too busy eating lasagne. Going to take an break now and start again in an hour.

• Fixed up AmazonSearcher so it passes its unit tests.
• Set search to redirect to the search form if no query is supplied.
• Created Searcher which holds all searchers and amalgamates results.
• Had lunch.

Hour Seven

Added a stylesheet so I can start to make it look like it should. Added in the concept of a User who may or may not be registered. Plus Location (get from IP initially, can be changed by the user). Everything is a bit haywire at the moment as I assimilate these changes.

Hour Eight

Had to take some time out to do some other things but now have a styled-up, eBay and Amazon searching app with a concept of User and Location. Upon initial visit you are silently logged in as a guest with a location guessed from your IP address. Eventually only sources that are relevant to your location will be searched.

I need to build the functionality to change your location and to register with the site. I also need to build in the concept of location to the existing eBay and Amazon searching libraries.

Still loads to do but lumbering towards a kind of “prototype” state.

Hours Nine and Ten

Last night I got the application running off the root of the site (localhost/ instead of localhost/finder/) and solidified a few architectural decisions.

This morning I’ve followed through on the architectural decisions and written any missing unit tests. There’s still one not passing to do with looking up location from IP address.

• Now works off site root, not /finder/blah
• Architecture of data sources, locations, etc. now clearer.

Hours Eleven and Twelve

Fixed up a lot of little bits and made prices/currency work how it should. Want to get on to persisting searches soon but lots of little things to do first.

• Fix failing unit test
• Sort results by price, lowest first
• Added concept of currency, extended Money from money module

Hours Thirteen and Fourteen

Made quite a lot of progress yesterday afternoon but then had to go out for a friend’s birthday. I’m not going to make 30 hours over the long weekend but it’ll be good to see how far I can get today. Not sure if I will try and make something live tonight or finish off the 30 hours another day – I’ll decide that later on.

• Make change location page work properly.
• Get Amazon and eBay searchers to respect user location and search the appropriate site.
• User registration.

Hour Fifteen – Half Way

Made good progress for an hour but had to stop working on it to produce a website at very short notice for the Lady Margaret short film.

I’ve used up half my allotted hours and the app is fairly functional. The big thing now is to add the persistent part of persistent search – to let users save their searches and be notified when new items become available. I also want to expand it to more data sources like abebooks.com. I’m not sure when I’ll get to do the other 15 hours but I definitely want to get something live quite soon.

• Added France, Germany, Japan, Ireland.
• Fixed for different currencies in Amazon searcher.
• Proper message if there are no results.
• Handle SearchErrors from Amazon.

I’ve revamped londoncinema.bluebones.net.

As well as a new design it has much-improved cinema and film pages, movie posters, trailers and imdb.com ratings.

Search is now even better because as well as taking your location it also uses imdb.com ratings to push better films higher up. With so many films showing it’s so easy to miss something good and with so many cinemas in London it’s so much better than trawling through cinema-by-cinema.

Not beautiful but it does effectively flush all cookies:

ClientCookie.install_opener(ClientCookie.build_opener())

I’ve added the most-requested feature to my Fixtures Generator.

You can now enter a newline-delimited list of team names in the initial form and get back team names in the fixture list.

Please let me know if you use it, especially if you have any problems with it.

Fixtures Generator

A formatting function to go from numbers like 333999333.33 to “333,999,999.33” in Haskell. Copes with negative numbers and rounds to 2 dp (easy to add a paramater for that should you wish).

Examples:

*Main> formatDecimal 44
"44.00"
*Main> formatDecimal 94280943.4324
"94,280,943.43"
*Main> formatDecimal (-89438.329)
"-89,438.33"

import Data.Graph.Inductive.Query.Monad (mapFst)
import List
import Text.Printf

formatDecimal d
    | d < 0.0   = "-" ++ (formatPositiveDecimal (-d)) 
    | otherwise = formatPositiveDecimal d 
    where formatPositiveDecimal = uncurry (++) . mapFst addCommas . span (/= '.') . printf "%0.2f" 
          addCommas = reverse . concat . intersperse "," . unfoldr splitIntoBlocksOfThree . reverse 
          splitIntoBlocksOfThree l = case splitAt 3 l of ([], _) -> Nothing; p-> Just p

If you know a simpler way or spot anything that should be done differently, please add a comment.

After investigating a lot of blind alleys I think I’ve solved a problem with pushing and pulling to and from a remote repository being very slow. darcs was issuing hundreds of scp commands just to push up a one line change. This seemed to be related to the _darcs/inventory file being a couple of thousand lines long (should be much smaller) on the remote repo.

In the final analysis my advice is very simple:

• Make sure you have recently done a darcs tag
• Issue a darcs optimize --reorder-patches command (the reorder patches part may be unnecessary and cause it to be much slower, but it worked for me although it took three hours).

This reduced push/pull time from 3 minutes to nearer 3 seconds.