There are a couple of possible approaches for the date line. At the moment we just run out of map and the 3x3 table contains a 2x3 map. Not good. Options:

1. Keep all tiles as they are and use SQL to calculate the joins.
2. Extend tile mosaic so that we have an overlap at either side.

I think (2) is going to be the more sensible option. Provided that we limit entries in atlas_location_to_entry to the tiles in the central block, wrapping etc will work with the existing code. Will need to:

a. prepare extra end panels down each side.
b. re-cut the tiles from this new panel grid.
c. check that the TileGenerator algorithm can handle wrapping tiles and apply it.
d. modify the atlas location linker and mark tiles as linkable/not linkable so that we know which ones are there for decoration only.
e. recreate atlas location links.

T
- posted by Tim Bruce @ 5:11 PM 0 comments

 
Back to the atlas. The overview map seemed wildly inaccurate as a navigation tool, which made it fairly useless. Stared at it for a while and realized that it's not actually a Mercator projection at all - the poles aren't stretched enough vertically. Tried a straight linear projection (multiplying both lon and lat by a constant) and that fits much better. We link each atlas (ie collection of tiles) to its own projection- when we've got more maps in then it'll use the database, but while we've only got two we're using a direct lookup for speed. There's a projection factory so it's all in one place.

Also improved consistency in TargetIndicator where it was using image sizes from both the database and the image itself. Now we're using database coordinates for everything.

Next things: make sure the rhumb lines are wrapping correctly and tidy up behaviour around the date line.

Index vacuum still running on w1 so I can't update the live jar yet.

T
- posted by Tim Bruce @ 12:36 PM 0 comments

 
Jsitetest passed and cvs archive tag is cannon_street. All aboard..

T
- posted by Tim Bruce @ 10:45 AM 0 comments

 
Swapped the order of the ath. login tests so we try the more-specific institution code before the more-general site-prefix.

Need to put the new jar live today if we're going to meet the Wednesday deadline for the client.

CVS committed, unit tested on the appserver. Now for jsitetest, release tag, release.

T
- posted by Tim Bruce @ 10:14 AM 0 comments

 
OK, I've made the required changes to the login process. If the main ath. login fails, we fall back and try their institution code under a similar lookup. Works fine here but will need more testing tomorrow before it goes live.

T
- posted by Tim Bruce @ 5:26 PM 0 comments

 
Back to work after a short DIY break.

Working on ath. authentication, as we have a selection of new accounts which share the same ath. prefix. We could set them up with this prefix as we do for all the others, but this would allow in everybody who shares the same prefix, and not all of them have bought.

So: each site does have a unique site id and I'd like to put those in as backup member ids for the ath. autologin. Testing is proving tricky because we've changed site IPs since we last dealt with this. Our user accounts are now updated (cheers Carl) but we still can't authenticate on the office servers. I've added a change request which will allow our new office IP to act as an athentication host.

I hope that once that's in, I'll be able to modify our client code to fall back on @ath.net. Then we can put the code live and add appropriate members for the new sites. Consulting API docs in the meantime...

T
- posted by Tim Bruce @ 4:29 PM 0 comments

 
Swapped the default map marker back to the ring now that accuracy is up. If you've got it, flaunt it. We've got it again.

T
- posted by Tim Bruce @ 6:31 PM 0 comments

 
Reverse mapping is in.

Now regenerating tile extents, will subsequently fire up an appserver somewhere and update atlas_location_to_entry accordingly.

T
- posted by Tim Bruce @ 6:27 PM 0 comments

 
The forward projection is in and works! This is with the elliptical correction. Next to get the reverse mapping in before regenerating the tile extents etc.

T
- posted by Tim Bruce @ 5:23 PM 0 comments

 
Examining the proj source code (free license / public domain) takes me back a few years to the joys of preprocessors and c/c++.

Architecture so far:

Application assembles a projection constants structure from command line arguments and defaults supplied in the code. The bits we need to worry about are the mercator projection in PJ_merc and the Clarke66 elliptical constants defined in pj_ellps.c

The ellipsoid is specified by the length of its semimajor axis (a=6378206.4, radius at equator) and the length of its semiminor axis (b=6356583.8, radius to pole). The code in pj_ell_set extends these into equivalent expressions of a and es (eccentricity squared, I think). From these, pj_init fills in other useful values (e, 1-es, 1/(1-es)) which get stored in the projection constants structure.

Using the very useful -E option in gcc to perform preprocessor substitutions and make the code more legible.

T
- posted by Tim Bruce @ 5:15 PM 0 comments

 
Found the gorgeous proj application on freshmeat, which maps coordinates into any system you care to name. Cue lots of trial-and-erroring, graphing differences in gnuplot and other fun stuff.

We have a match.

I think they're using Mercator projection with an elliptical distortion known as Clarke 1866. Everything lines up to within 3 Mercator meters. This is quite a relief.

Just for comparison, this is the full list of elliptical forms supported by proj:

MERIT a=6378137.0 rf=298.257 MERIT 1983
SGS85 a=6378136.0 rf=298.257 Soviet Geodetic System 85
GRS80 a=6378137.0 rf=298.257222101 GRS 1980(IUGG, 1980)
IAU76 a=6378140.0 rf=298.257 IAU 1976
airy a=6377563.396 b=6356256.910 Airy 1830
APL4.9 a=6378137.0. rf=298.25 Appl. Physics. 1965
NWL9D a=6378145.0. rf=298.25 Naval Weapons Lab., 1965
mod_airy a=6377340.189 b=6356034.446 Modified Airy
andrae a=6377104.43 rf=300.0 Andrae 1876 (Den., Iclnd.)
aust_SA a=6378160.0 rf=298.25 Australian Natl & S. Amer. 1969
GRS67 a=6378160.0 rf=298.2471674270 GRS 67(IUGG 1967)
bessel a=6377397.155 rf=299.1528128 Bessel 1841
bess_nam a=6377483.865 rf=299.1528128 Bessel 1841 (Namibia)
clrk66 a=6378206.4 b=6356583.8 Clarke 1866
clrk80 a=6378249.145 rf=293.4663 Clarke 1880 mod.
CPM a=6375738.7 rf=334.29 Comm. des Poids et Mesures 1799
delmbr a=6376428. rf=311.5 Delambre 1810 (Belgium)
engelis a=6378136.05 rf=298.2566 Engelis 1985
evrst30 a=6377276.345 rf=300.8017 Everest 1830
evrst48 a=6377304.063 rf=300.8017 Everest 1948
evrst56 a=6377301.243 rf=300.8017 Everest 1956
evrst69 a=6377295.664 rf=300.8017 Everest 1969
evrstSS a=6377298.556 rf=300.8017 Everest (Sabah & Sarawak)
fschr60 a=6378166. rf=298.3 Fischer (Mercury Datum) 1960
fschr60m a=6378155. rf=298.3 Modified Fischer 1960
fschr68 a=6378150. rf=298.3 Fischer 1968
helmert a=6378200. rf=298.3 Helmert 1906
hough a=6378270.0 rf=297. Hough
intl a=6378388.0 rf=297. International 1909 (Hayford)
krass a=6378245.0 rf=298.3 Krassovsky, 1942
kaula a=6378163. rf=298.24 Kaula 1961
lerch a=6378139. rf=298.257 Lerch 1979
mprts a=6397300. rf=191. Maupertius 1738
new_intl a=6378157.5 b=6356772.2 New International 1967
plessis a=6376523. b=6355863. Plessis 1817 (France)
SEasia a=6378155.0 b=6356773.3205 Southeast Asia
walbeck a=6376896.0 b=6355834.8467 Walbeck
WGS60 a=6378165.0 rf=298.3 WGS 60
WGS66 a=6378145.0 rf=298.25 WGS 66
WGS72 a=6378135.0 rf=298.26 WGS 72
WGS84 a=6378137.0 rf=298.257223563 WGS 84
sphere a=6370997.0 b=6370997.0 Normal Sphere (r=6370997)

Next thing is to implement that datum in our HCProjection class. Two days to go until I'm on holiday for the new house. Woohoo!

T
- posted by Tim Bruce @ 5:51 PM 0 comments

 
Now to apply our shiny new gnuplot tools to work out whether UTM approximates our gazetteer any better than the straight ln(tan (pi/4 + lat/2)) mercator.

T
- posted by Tim Bruce @ 2:14 PM 0 comments

 
Art remark is underway.

Cryptic comments obtain.

T
- posted by Tim Bruce @ 2:03 PM 0 comments

 
Looks as if the Bm art book didn't get its links remarked after their regeneration last week - there are loads of lovely links sitting in the person_entry_to_entry table, but none in the main links table. Will remark it once an in-progress index merge is complete.

T
- posted by Tim Bruce @ 11:51 AM 0 comments

 
Citations now working properly - there's an svCitationExtras string which we declare in setup-citation.inc and you can use to append extra parameters to the citations.

T
- posted by Tim Bruce @ 11:16 AM 0 comments

 
Charts now appear on pp. Commands are stripped.

Citation for charts still needs work.

T
- posted by Tim Bruce @ 5:20 PM 0 comments

 
secid handling now in separate include file. I've changed the logic to invoke this earlier on in entry.jsp so that the secid is already correct when we come to prepare the printer-friendly link.

Citation link looks OK.

T
- posted by Tim Bruce @ 5:12 PM 0 comments

 
printer-friendly dtables just about there - need to:

1. split off secid-modification into include file.
2. handle chart parameter.
3. don't display options around charts.
4. check citation accuracy.

T

- posted by Tim Bruce @ 1:03 PM 0 comments

 
Can't leave atlases as they are, so I've reverted to the square marker as the default and kicked off a relink of atlas locations against the new tiles. That way we've at least got a rough demo system until we work out the oddities in the projection.

T
- posted by Tim Bruce @ 12:21 PM 0 comments

 
In the meantime, this is what's on the cards for tomorrow:

1. printer-friendly dtable and chart entries.
2. find out why b. art. is still short on xrefs.
3. profile and speed up atlas handling.

Then we shall have weekend. Oh yes.

T
- posted by Tim Bruce @ 5:44 PM 0 comments

 
Not much luck with the projection. I've checked using gnuplot and the relationship between our calculated latitude and gazetteer latitude is definitely not linear. We line up at the equator (of course we do, everything's zero) but then we get higher than the gazetteer up to about 35 degrees, before coming back into alignment around 60 degrees and falling rapidly below. This isn't good. The distances involved aren't huge when you see it on the screen - we're on the right 200x200 pixel tile almost every time - but I do worry that it will make street-level maps unusable.

Going to have to ask the supplier for more detail on their projection.

T
- posted by Tim Bruce @ 5:39 PM 0 comments

 
The new markers are highlighting the inaccuracy of our projection - London's latitude is bang on, but up to the north our circles are a bit too low. The equator's good, south america's pretty good - maybe a bit low. Will tweak the projection to try to line everything up.

T
- posted by Tim Bruce @ 11:29 AM 0 comments

 
The atlas element in RML now sports a "marker" attribute, with options:

r - ring
s - filled square
c - crosshair

"r" is the default if none other is specified.

Preview site is up to date.

T
- posted by Tim Bruce @ 4:46 PM 0 comments

 
Added cross-hair marker to TargetIndicator, also got IndexingParser to record the presence of atlas elements in the meta field so we can search for atlases and indicate them on the results pages.

T
- posted by Tim Bruce @ 3:32 PM 0 comments

 
Been adding machinery for alternative target markers. There's now an IMarker interface which allows you to plug your own marker into the TargetIndicator, or you can choose to use one of the existing set. These contain the original square marker with drop shadow and a new ring+shadow marker.

First attempt to render a ring using concentric ellipses gives nasty moire patterns where the ellipses don't touch. Now creating a proper polygon to represent the ring and getting the shadow by translating it a few pixels. Looks OK.

Next to add the specification attribute to RML.

T
- posted by Tim Bruce @ 3:00 PM 0 comments

 
The import and atlas processing for the full test book is complete. Seems to be working fine on first inspection, more in-depth testing to come.

Modified the rendering code so that it will work around missing "at" attributes. This allows us to show atlases on the preview site (without the interactive element) even though they've not yet been through the location linking process. Deployed to preview site.

T
- posted by Tim Bruce @ 12:04 PM 0 comments

 
The biggie - importing a test book with the complete dataset.

T
- posted by Tim Bruce @ 6:04 PM 0 comments

 
Added a new method to AtlasDAO which finds a tile id from an AtlasLonLat - stripped down and much faster than fetchAtlasView. Atlas linking now tolerably fast but could still do with tuning.

T
- posted by Tim Bruce @ 5:35 PM 0 comments

 
new attributes added to atlas element: "nav" and "labels". Both default to false, but you can specify a value of "true" to turn on navigation arrows and lon/labels respectively.

Modified test book (western europe) accordingly, reimporting.

T
- posted by Tim Bruce @ 5:04 PM 0 comments

 
Added the you-are-here. It's in there as atlas number 2, so you just have to reference it in the rml as you would with the main atlas. It's interactive - you can click on the minimap and it'll take you to the nearest entry.

Fixed a couple of integer overflows which were cropping up in TargetIndicator as result of fitting the whole world into 180x90.

Hope that the projection is close enough for the minimap to be useful - my test data set is a bit too small to see properly.

Next: add attributes to the atlas element which allow you to turn on/off the navigation and lat/lon displays. Then try to speed up the atlas link step and import a larger test book.

T
- posted by Tim Bruce @ 4:11 PM 0 comments

 
AtlasRenderer now inserting lon/lats in proper format (N/S/E/W as appropriate).

Next to add the "you are here" display of the whole world. A microscopic dot on a microscopic dot - all driven by a small piece of fairy cake.

T
- posted by Tim Bruce @ 2:28 PM 0 comments

 
Added lon/lat display around the border of the atlas. Need to add N/S/E/W indicators, rather than +/-.

T
- posted by Tim Bruce @ 12:32 PM 0 comments

 
Navigation in all directions is now working, but raises a design decision: what should happen when you're at the top of the map and click North? There are several possibilities:

1. stay where you are
2. loop around to the bottom of the map at the same longitude so that subsequent hits on North will take you back where you started.
3. leap over the pole and arrive at the top of the map, 180 degrees of longitude away on the other side of the world.

3 might make the most sense in terms of navigation, but it could be confusing. Repeatedly clicking North will alternate between two tiles at the same latitude but 180 apart in longitude.

The problem arises because at the north pole it's impossible to go any further north - we don't have the pole to show so we've got no framework in which to tell the user that. Option 3 isn't really any more "accurate" as your destination is probably at exactly the same latitude as your starting point.

The same problem arises with north-east, north-west etc. I've gone for option 2, where the indicated directions are accurate for all but the north/south boundary cases and you can cycle around the whole earth by following the same direction. Will take user feedback and see what people think.

T
- posted by Tim Bruce @ 12:17 PM 0 comments

 
Hashmap-based navigation is in and working. Added another hashmap to handle wrap-arounds at the map edges.

North, south, east and west all working fine and wrapping properly.

Now to add the other four directions.

T
- posted by Tim Bruce @ 11:01 AM 0 comments

 
Created a full enumerated type to represent direction. This will allow us to simplify the construction of the ocean-leaping sql by using the directions as keys into a hashmap.

T
- posted by Tim Bruce @ 10:28 AM 0 comments

 
Ocean-leaping works! North, south, east, west working. Other four on Monday. Plus streamline to API, which at the moment gives you the atlas tile id from which you have to work out the entry id. Much simpler just to return the entry id.

Sweet!

T
- posted by Tim Bruce @ 5:21 PM 0 comments

 
Altered the db schema again so that we now track atlas tile ids in the atlas_location_to_entry table. This should allow us to the reduce the ocean-leaping problem to an sql query.

Reimported the sql, repopulated the atlas tile table and refilling the atlas_location_to_entry table. Once that's done I'll put the scrolling system into place.

Modified the again so that it's finding the centre tile at run time. (ie allows regeneration of atlas tile with potentially different ids). We now have both methods in place so we can test one against the other in the profiler.

T
- posted by Tim Bruce @ 4:34 PM 0 comments

 
Map navigation now in place - all eight directions available, but at the moment there's no filter to warn you that there's nothing to go to if you're on the coast. The solution we discussed will leap oceans/lakes/mountains and take you to the next feature in your chosen direction. Hmm. Will think about that this afternoon.

Pretty artwork required for arrows!

T
- posted by Tim Bruce @ 12:21 PM 0 comments

 
Looked into IP autoloin issues discovered by Claire. If a client has an existing session (but has not logged in) when we add IPs for them, they remain logged out until their session is invalidated/expired. Will delay work on the login logic until we have the new login page later this month. Then sort out the unholy mess which is login.inc.

Modified key rendering in hawkins to indicate that there is an extended key available.

T
- posted by Tim Bruce @ 11:32 AM 0 comments

 
Implemented AtlasBlock, which implements IRenderable and defers all the heavy stuff until render() is called. AtlasBlock calls on AtlasRenderer for the main work, and rendering seems acceptably fast. Will look to tune it before release.

Got AtlasBlock to render into a form and added code to handle form submission on entry_at.

Now you can click on the map - the system will recognize the nearest location and show you the appropriate entry. Swweet. The book itself needs content to make more of the entries, but it's looking promising.

entry_at.jsp is accessing AtlasDAO directly for the reverse lookup - we'll want to integrate that with the standard QueryTool for pooling.

Tomorrow to add the atlas scroll buttons.

T


- posted by Tim Bruce @ 5:45 PM 0 comments

 
Location processing complete on test vol.

Now to:

1. implement the AtlasBlock analogue to the ImageBlock
2. handle atlas clicks
3. handle atlas scrolling
4. tune location processing
5. add location processing to the import sequence

T
- posted by Tim Bruce @ 2:28 PM 0 comments

 
The processing of the test volume is fairly slow - only 650 entries processed in about 25 minutes. There are 1,500 entries in the test volume, but the full atlas will contain about 90,000. Will have a look at speeding this up before the final import.

T
- posted by Tim Bruce @ 12:15 PM 0 comments

 
Hooked together the atlas location processing, from a test script to the VolImport manager, to the parser and the entry bean.

All seems to be working OK. There's a new home method on the entry bean which clears the link table for all entries from a specified volume, then we iterate through the entries and insert rows into the atlas_location_to_entry table. We also modify the entry rml to add a new "at" attribute to the atlas element, tracking the appropriate atlas tile id.

T
- posted by Tim Bruce @ 12:02 PM 0 comments

 
Reintroduced operations tally to stats files and re-published them.

Integrated atlas sql scripts into a single file, modifying the projection table so we can hook it up with the AtlasProjectionFactory. Re-applied script to db_dev for testing.

Modified TileGenerator so it outputs the tile-generation instructions to our SQL log. Re-ran to test.

Continued work on atlas rml import, filling out the methods on EntryBean which manage the back-links from location to entry and beginning to implement AtlasLocationParser. Thinking about the display side, we may be able to use an ImageBlock-like object to defer all db interaction until the rendering stage. Will check it out once import process is completed.

Off now for a final look around Churchfield before contracts are signed!

T

- posted by Tim Bruce @ 5:04 PM 0 comments

 
began to integrate the two-stage atlas resolution process. The VolImport bean now has a method to coordinate things, just need to add appropriate support methods to EntryHome and Parser.

Revised the mansion house release twice, once to fix a code path which tried to use ISBNs without retrieving them first (oops) and the second to revert from our new meta boost of 30 to our old one of 7. This is at the request of the US sales team.

T
- posted by Tim Bruce @ 6:17 PM 0 comments

 
Cluster update is complete and the site is running with the new jar.

T
- posted by Tim Bruce @ 1:03 PM 0 comments

 
This month's release (Mansion House) is now up on www1 and seems fine. It's passed jtest and jsitetest. Started the getimage, ready to update the cluster after lunch.

T
- posted by Tim Bruce @ 12:46 PM 0 comments

 
Decided to split the atlas element handling in two for efficiency. We'll pre-calculate the appropriate atlas tile and insert new attributes, in a similar way to the x element handling.

Temporarily commented out while importing the test volume.

T
- posted by Tim Bruce @ 5:38 PM 0 comments

 
Created a small test atlas with 1500 entries around europe. Will use this to test navigation and develop the cell-by-cell movement.

T
- posted by Tim Bruce @ 4:08 PM 0 comments

 
Added getAllPublishers, which returns an array of IPublisherDescriptors for all content available to the calling member. The publishers are sorted alphabetically by name.

T
- posted by Tim Bruce @ 2:41 PM 0 comments

 
Added getVolByPublisher method to the api. This follows the form of the getVolBySubject call, returning an IPublisherPage from which you can obtain publisher information and an array of IVolDescriptors. The list of volumes is trimmed to those available to the member calling the api.

T
- posted by Tim Bruce @ 2:27 PM 0 comments

 
Added getISBN to IVolDescriptor and VolBean. Imported existing ISBNs from subscription db and modified import process to pick up the isbns of incoming rml.

T
- posted by Tim Bruce @ 12:09 PM 0 comments

 
Added getPublisherName to IEntryInfo, which returns the full human-friendly name of the publisher.

T
- posted by Tim Bruce @ 4:54 PM 0 comments

 
Started API additions for next week's release - these are requests for Carl.

Added isVolAvailable and isEntryAvailable.

T
- posted by Tim Bruce @ 4:37 PM 0 comments

 
Admin system: fixed deletion of referers where the referral url contains ampersands.

This is the last item on the list - now putting the new system live!

T
- posted by Tim Bruce @ 3:31 PM 0 comments

 
Admin system: now handles duplicated IP addresses for the same member. It no longer crashes, instead the duplicates are weeded out and the user receives a nicely-sorted list of those which remain.

T
- posted by Tim Bruce @ 12:56 PM 0 comments

 
Admin system: now catches duplicate referers and gives an appropriate warning message.

T
- posted by Tim Bruce @ 5:03 PM 0 comments

 
Admin system: all pages now have proper page titles.

T
- posted by Tim Bruce @ 3:13 PM 0 comments

 
Admin system: client name edit box now gets focus on the client details page.

T
- posted by Tim Bruce @ 2:30 PM 0 comments

 
Put the reverse-lookup live on xplus. Next steps with the mapper:

1. create a test book
2. import it
3. autogenerate location-to-enty links
4. create cell-by-cell navigation system
5. pass spec to content
6. update office and live dbs appropriately
7. import book
8. tweak projection for improved accuracy
9. get media site to host target generation jsp
10. test

I'm going to spend this morning fixing an accumulated list of bugs and feature requests for the admin system.

T
- posted by Tim Bruce @ 10:25 AM 0 comments

 
Hooked up a reverse-lookup to the new location table through the atlas DAO. Now you can click on the map and we'll give you the name of the nearest feature with a map centred on your click. Progress!

T
- posted by Tim Bruce @ 5:40 PM 0 comments

 
Added a reverse mapper to the TargetIndicator, which can go from a tile id and coordinate to a LonLat.

Modified the atlas test page so that tiles are presented on a form. Added code to harvest tile id and coordinates from a form submission, reverse map them using the TargetIndicator and re-centre the map on the new coordinates.

T
- posted by Tim Bruce @ 5:13 PM 0 comments

 
Added new fields to the atlas tiles so that they track their own pixel width and height.

T
- posted by Tim Bruce @ 4:41 PM 0 comments

 
Added a reverse-lookup table to the database so we can navigate from lon/lat back to location and test efficiency. Seems to be speedy enough.

T
- posted by Tim Bruce @ 3:56 PM 0 comments

 
Added support for a new "atlas" element in rml. Presently, it has parameters for atlas id, longitude and latitude although this list is subject to change. The preview site will now render an appropriate view of the atlas.

T
- posted by Tim Bruce @ 2:24 PM 0 comments

 
Amended stats generation process to handle high-ascii correctly. We maintain the user's search term up to the final stage, then process it to replace high ascii chars with an escaped unicode entity reference. By the time it's been through the template toolkit, it ends up as a character reference which is resolved by the browser.

Sounds complicated. Looks all right.

T
- posted by Tim Bruce @ 4:48 PM 0 comments

 
Joined the split tiles up,so finally we can look at London!

T
- posted by Tim Bruce @ 12:28 PM 0 comments