Basic tomcat installation succeeded on orac - it's serving the test applications fine. Bonza!

I'm reading the tomcat book in parallel and will make notes here of things to look at in the future.

Number 1: We could use the JNDIRealm authentication plug-in to authenticate office users against our master LDAP server. This means that the admin system, content systems etc wouldn't need to track their own login credentials and we could centralize the administration.

T
- posted by Tim Bruce @ 11:58 AM 0 comments

 
Time for another crack at Tomcat.

I'd like to get it configured so that we can use it as a drop-in replacement for jserv. We've stayed with our initial architecture for a long time because it works - I did try Tomcat on one server a couple of years ago, but it couldn't handle the load and just grew more and more threads before slowing to a crawl. The trouble with jserv is that it's no longer updated and supports only an old version of the servlet / jsp spec. In particular, we can't use taglibs like JSTL or our own.

jserv allows speedy recompilation of pages after in-place editing. This means that someone from content can see the effect of their edits straight away, without waiting for a developer to package up and deploy a .war file. Tomcat will need to support that.

So, on with the download!

T
- posted by Tim Bruce @ 10:13 AM 0 comments

 
Modified search.jsp to default to longer results for US trial account.

T
- posted by Tim Bruce @ 5:43 PM 0 comments

 
James working on the atlas page polish.

After conversation with Carl, going to hardcode initial search preference for us trial account on jsp. If it becomes widely-requested then we'll look at it again.

T
- posted by Tim Bruce @ 5:07 PM 0 comments

 
Spent the morning at a "tech talk" from BEA about webservices and SOA. Looks interesting, though to some extent just another cycle in the new-acronym-which-delivers-code-and-object-reuse saga. Eventually we'll have a framework so abstract that it requires its own scripting language and then we're back to where we started. Still, a worthwhile heads-up and my head is now fimly up.

As for the HP bit about their management system, I'm not saying anything. I think we can do without.

New login page is going live.

T
- posted by Tim Bruce @ 3:51 PM 0 comments

 
Investigated the possibility of making the US trial favour (favor ?) longer results by default. Won't be easy - the language preference is always in effect and so we don't have to handle interactions with user selection. The metasearch preferences, however, should be overridable by the user in the usual way. This means we either have to create a new client field to store them or beef up the metasearch logic to know which choices are mutually exclusive and decode the clashes when required. I suspect the latter is the best option.

T
- posted by Tim Bruce @ 5:25 PM 0 comments

 
Profiling the atlas retrieval code hasn't thrown up any horrors - quite a few chars are getting allocated in the sql calls but there's not much we can do about that.

Indexing the atlas entries seems OK. They're all accumulating "atlas" in the metadata field, so we can look at adding a new rank option to prefer atlas entries.

Been thinking about how to integrate a wildcard atlas entry, which will display the map at any given lat/lon. The display side would be fairly easy, but the integration itself is trickier. Would the entry itself hold the form code? If not, and the form is on a jsp, how would the web server know which entry to request without a nasty bit of hardcoding? We could give it a particular category (like we do for jacket pages), but then we still need to know which volume to use. If the result is displayed as an entry page, would we want the entry title to reflect the requested coordinates? Alternatively, we could bypass the entry system entirely and have all the logic in a jsp, making appropriate requests so that the logs reflect usage of the atlas data.

I think the best solution would be to have an RML element to handle it in a similar way as we do for atlas-click navigation.

How badly do we want this feature? Have to say I'm quite keen to get off atlases and on to the next big thing asap.

T

- posted by Tim Bruce @ 5:02 PM 0 comments

 
Those missing stats turned up when I re-ran the stats generator and are now live. Can't see any reason why they should have failed first time - perhaps the generator was interrupted? Watch carefully around the 3rd of May.

T
- posted by Tim Bruce @ 12:22 PM 0 comments

 
Test book imported and processed. The key resolver seems to be working fine and the UI attribute is doing what we want.

IDisplayPreferences handling has been improved and is now much neater.

I've applied the sql update to our office and live servers so Amit can put up an up-to-date import jar.

I think we're just about done. Performance tweaks and integration of pretty HTML tomorrow.

T
- posted by Tim Bruce @ 5:34 PM 0 comments

 
Test book is ready for import. We're juggling the atlas attributes as in the mail below.


Hi,

Just to document what we discussed, please shout if I've missed anything.

The atlas element currently features a couple of display attributes:

nav="true" - enables navigation arrows
labels="true" - enables lon/lat figures around edge of map

We're adding more objects to the display - a frame, a selection of map sizes and a link to the key if available. Rather than adding attributes to govern each of these, we're doing away with the two above and just going with:

ui="true" - enables navigation arrows, lon/lat labels, map size controls, frame and link to the key
key="1234" - local entry id of key entry. Skip to omit link.

T
- posted by Tim Bruce @ 4:38 PM 0 comments

 
Need to refactor the IDisplayPreferences handling so that it gets passed in getAsHTML rather than setText.

T
- posted by Tim Bruce @ 3:41 PM 0 comments

 
The key resolver is in there now, running on the back of the atlas location linker. I'm going to create a smaller test book with the new complement of attributes for testing.

T
- posted by Tim Bruce @ 2:36 PM 0 comments

 
Rough final atlas page is working. We display a link to the key (if supplied), text links for map size options (large and small) and a frame around the whole thing. You only see these if you specify ui="true" on the atlas element. We may wish to condense some of the other options (labels, nav) into the ui attribute as well.

Page design is ready for James to spiff it up. I need to work on the key resolver, which will insert a gkey attribute when it sees a key attribute, mapping local entry ids to globals in the same way as our internal linker.

T
- posted by Tim Bruce @ 12:02 PM 0 comments

 
Will get final atlas page design up and running today. I'll put a rough but fully-functional page together, ask James to prettify it and then incorporate his changes.

T
- posted by Tim Bruce @ 11:06 AM 0 comments

 
Stats are missing for client 718 in March 04. There are entries in the db for operations in that period, but no intermediate xml report. Re-running stats generation script to see if that fixes it.

T
- posted by Tim Bruce @ 10:55 AM 0 comments

 
Added handling of "ui" attribute on atlas element, which causes the page to display the map-size controls when true. Modified the rendering of these controls so they're hyperlinks rather than buttons. A link to the key is next.

Jboss 3.2.3 up and running on local machine, will try running the admin system on it next week.

Ant 1.6.1 is out and seems to have extended support for jboss chores - perhaps auto-creating deployment descriptors? Would be good to revise our toolset a bit: latest versions of ant, lucene and jdk plus finally get rid of jserv in favour of tomcat. It must be faster by now and we're not serving as many pages as we were on the free site.

T
- posted by Tim Bruce @ 5:17 PM 0 comments

 
Stats system modifications complete. We now host a page which informs the user that no statistics have yet been recorded for their account.

Will update the stats site once the cluster update is complete.

T
- posted by Tim Bruce @ 5:01 PM 0 comments

 
Cluster update in progress.

Working on modifications to pubstats report script which will improve the handling for "empty" clients - those who have accumulated no stats for a given month (or at all). Perl documentation seeing some heavy use.

T
- posted by Tim Bruce @ 12:08 PM 0 comments

 
Index merge complete on the live site. Cluster update would be good, so I'm running a preparatory getclient tonight to speed things up tomorrow.

Work on the new login page is complete. It handles username/password logins as well as library cards - we'll need to add a rewrite rule to the apache config when it goes live so that we can remove the old library card login page.

T
- posted by Tim Bruce @ 5:17 PM 0 comments

 
sql import complete for mos teeth. Index merge now in progress.

Still working on the new login page.

T
- posted by Tim Bruce @ 10:56 AM 0 comments

 
bAnonymousUser has now gone, though this one took some doing. Remember, kids, awk uses = and == in the same way as c. Don't silently change all rows so they match your test criteria then freak out when you see the result...

T
- posted by Tim Bruce @ 5:37 PM 0 comments

 
bPageCaching bites the dust.

T
- posted by Tim Bruce @ 3:43 PM 0 comments

 
sql update for mos teeth in progress on w1.

Streamlining code as it is transplanted into the new login page. We've accumulated a few variables over the years which are not necessarily used any more - best to clean up as much as possible. bIsPaidSite is the first casualty.

T
- posted by Tim Bruce @ 3:28 PM 0 comments

 
Index merge completed overnight in the office, so Mos Teeth is now available for testing. Briefly disconcerting to find that no entry extracts were being displayed until I discovered that the vol was marked "No extracts" in the db. Now fixed. SQL and indexes on the way to the site.

T
- posted by Tim Bruce @ 12:24 PM 0 comments

 
Added basic login support to the new login page. Tomorrow we'll get it to handle library-card logins too.

T
- posted by Tim Bruce @ 6:37 PM 0 comments

 
Mos teeth images are live. Nice.

T
- posted by Tim Bruce @ 6:10 PM 0 comments

 
Mos teeth looks to have linked correctly, now merging indexes and putting media live. Search thumbnails are done.

T
- posted by Tim Bruce @ 5:32 PM 0 comments

 
Tracking the problem with tile generation, it looks as if the tiles which went awry shouldn't have been there in the first place - the database reckons our right-most tiles are in column 305, and they are fine. So I won't lose any more sleep about that.

3. Done

T
- posted by Tim Bruce @ 4:56 PM 0 comments

 
2. Atlas navigation now using the spiffy new graphics.

T
- posted by Tim Bruce @ 4:26 PM 0 comments

 
OK, some progress:

1. Stats culled. This got a bit in-depth, comparing bits of logs against email activity against salesforce records. We've found a sample entry which made it half-way through the system before disappearing. SF are checking their logs.

6. SQL update complete.

T
- posted by Tim Bruce @ 4:07 PM 0 comments

 
I'm back! Did you miss me? Given the readership of this blog: did I miss me?

Provisional list of things to tackle this week:

1. Cull stats from the logs to compare with the week-long trial requests.
2. Integrate new atlas navigation images.
3. Fix problem with atlas tile generation for east end of map and regenerate.
4. Find out what's happening with blank clients in the stats system: they should at least get a "nothing viewed" page.
5. Check import of mos teeth and send sql to site.
6. Run this afternoon's admin sql update while Matt's at the dentist.
7. Review other new content ready for import.

Bet there's more to add by the end of the day!

T
- posted by Tim Bruce @ 12:28 PM 0 comments

 
Other tile joins look fine.

Codebase is checked in and autobuild works fine.

Will integrate new login page on Monday I'm back - it's not going live next week.

T
- posted by Tim Bruce @ 4:58 PM 0 comments

 
Glueing atlas tiles together has shown up oddities in column 306 out to the end (col 379). Looks as if all tiles are full panel size - no cropping has taken place. Maybe an integer overflow in the perl script? Can edit and re-run it for those tiles.

T
- posted by Tim Bruce @ 4:36 PM 0 comments

 
Mos teeth import looks to be going well so far. It's onto the link stage.

T
- posted by Tim Bruce @ 3:03 PM 0 comments

 
Blah. Just hit the wrong button and lost a long post.

In summary:

After conversation with Carl, we've agreed that

1 map size controls to come from RML renderer
2 their effects to be handled by entry.jsp
3 new parameter on atlas element to request map display controls
4 new parameter on atlas element to link to map-key entry

I'm off next week (moving house) so there are a number of things to fit into this afternoon:

1. Mos Teeth
2. Glue atlas tiles together
3. Add code to new login page
4. Read outstanding material and clear desk
5. Try jboss 3.2.3
6. Bring atlas code up-to-date and check in

T
- posted by Tim Bruce @ 2:36 PM 0 comments

 
Right then: DisplayPreferences are now implemented across the system. This will require a little modification to everything which calls retrieve or retrieveByVolCategory for the next jar update.

The atlas rendering system now inserts buttons for map size above each map - this needs some design work, both in how it works and how it looks. Do we want the choice on each map? Do we want a preferences page for this and other display options? Discussion to come. For the moment, the entry page looks for form submission and updates its session's IDisplayPreferences object appropriately before regenerating the page.

T
- posted by Tim Bruce @ 4:40 PM 0 comments

 
Found the problem with mos teeth at last: the alt text for an image included entity references and it turns out that's not allowed - not for entities from the external subset, at least, and that includes all of ours. Content will edit the file.

Image slicing is complete, need to glue the joins between panels.

T
- posted by Tim Bruce @ 12:01 PM 0 comments

 
Started to add the atlas preferences system (large map, small map). To avoid an ever-growing list of parameters on the retrieve API, there's now a new IDisplayPreferences interface which will collect all these things together. The client will obtain an IDisplayPreferences object through a factory (which also offers a default implementation) and pass it with each retrieve request. We want the fields of this object to be relatively long-lived so that a users session can cache a single idp object and pass it with each call.

Added support from the API through to the ContentDAO - now to integrate it with the parser.

T
- posted by Tim Bruce @ 4:15 PM 0 comments

 
Ghost highlight was down to an error with tiles which span the dateline - the target was going between them on the "outside" rather than the "inside". Life gets complicated when east eventually becomes west...

Now fixed.

T
- posted by Tim Bruce @ 12:09 PM 0 comments

 
test import taking a long time (it's running on the image-slice machine) so I'm going to look at more atlas stuff in the meantime. Along the dateline we're getting a ghost highlight which doesn't align east-west with the real one. Will track it down.

T
- posted by Tim Bruce @ 11:02 AM 0 comments

 
Started a test import of mos teeth on my local appserver, going to db_dev. I've added more output to the parse exception handler so we can find out where the error's coming up.

Very keen to apply the atlas sql changes across the board so that I can put up a new import jar. Until all databases are the same shape, we have to stick with what we're running.

T
- posted by Tim Bruce @ 10:58 AM 0 comments

 
Atlas location link is complete, so everything on the map knows where it is. Image slice still in progress.

Next thing is to find out what happened to mos. teeth.

T
- posted by Tim Bruce @ 10:23 AM 0 comments

 
AtlasDAO will now hunt in succesively wider radii when you click on the map. It used to look for a location within 1 degree - now it iterates up to a radius of 90 degrees, so even a click in the middle of the Pacific will get you something (provided the associated atlas is well-enough populated).

Tile regeneration is progressing steadily, so we get little patches of image on an otherwise blank map.

T
- posted by Tim Bruce @ 4:18 PM 0 comments

 
The office appservers are now forgetting how to serve web pages quite frequently (giving directory listings instead). Downloading jboss 3.2.3 for evaluation. We're currently using 3.0.3.

T
- posted by Tim Bruce @ 3:50 PM 0 comments

 
Modified AtlasDAO so that it knows only to hunt for linkable tiles when preparing an atlas view.

T
- posted by Tim Bruce @ 3:12 PM 0 comments

 
Fixed a couple of login problems with the stats system.

T
- posted by Tim Bruce @ 1:11 PM 0 comments

 
location link in progress.

Image slicing ran out of steam on Saturday due to disk quotas. Now re-running in the shared area (rather than my home). Duh!

Tooth import failed with a message about image tags out of context. Can't see why yet.

T
- posted by Tim Bruce @ 12:23 PM 0 comments

 
TileGenerator is flagging tiles correctly. Modified the location link step to link only to flagged tiles. Re-running the location link process on readingtest.

Started import for mos. teeth.

T
- posted by Tim Bruce @ 11:41 AM 0 comments

 
TileGenerator is now aware that some tiles should be linkable, others not. It works out the east and west boundaries of the projection and only marks a tile as linkable if it falls within that range. The input to TileGenerator is in projected units, so we can just feed it the appropriate out-of-bounds paramaters and it should all work.

Testing.

T
- posted by Tim Bruce @ 10:46 AM 0 comments

 
Added "linkable" field to AtlasTile. Now to update TileGenerator to set it accordingly.

T
- posted by Tim Bruce @ 4:36 PM 0 comments

 
Amended atlas sql script to reflect new features, including "linkable" field for atlas tiles.

Reapplied script to db_dev, so we're building from nothing again. Should be the last time - after this we can apply changes to db_server and the site.

T
- posted by Tim Bruce @ 4:09 PM 0 comments

 
Image slice script updated and now chopping away.

T
- posted by Tim Bruce @ 12:22 PM 0 comments

 
A few things:

1. Tabs Med was missing a few entries' indexes because of rougue > s in the rml. Content are taking another look.
2. Finally figured out the command line parameters to imagemagick's crop command. To get the left-most 1000 pixels of an image, use

convert -crop -[image width - 1000] image out

and to get the right-most, use

convert -crop [image width - 1000] image out

The WxH+x+y notation is misleading - width and height don't correspond to the desired result of the crop. Instead you're specifying a rectangle which will be intersected with the image. The result will be the intersection. Easy once you know that. Tough when you don't!

3. Rearranged atlas panels and added padding either side ready to re-cut the tiles.
4. Big discussion coming up about sections, default behaviour, show all / show default. Notes so far are below.
5. Glastonbury ticket service need more phones.

The section problem: If you look at the Colum Enc entry on Siberia, you'll see a paragraph at the top with clickable headings below. There is no "Show full entry" link. That is the problem, below is where I think we stand:


Right then - I think this might get complicated. The tech discussion is
below, but the key thing to resolve is: what do we _want_ to happen?

Those sections are marked with s="c", ie default state=closed.

The primary effect of this is that the sections don't open up when fed a
section path of "-". They have to be given an explicit id number on the
section path to respond.

The code discounts the fact that these sections are closed when deciding
whether to show the "Show full entry" link. This is because the sfe link
just feeds in a ".-" section path, which these sections won't respond to
anyway.

If we change this behaviour then we have to have parallel notions of
"Default" and "Full" view of an entry. I think that's possible, but it
will be a source of more confusion. For example, we've already decided
that we won't display all tabs open simultaneously in a tabbed entry -
so a full view won't always be a full view.

I think the problem arises because of the way those entries are
structured. We should talk to Phil and James and see what they think. As
I understand it, the intent is to have a structure like this:

[stuff always displayed] (A)
[heading] (B)
[heading] (C)
[heading] (D)

where the headings B, C and D are clickable to display the associated
text but A is always visible.

The way the entry is structured right now, A is in the root section
and B, C and D are first-level children. We've got this s="c" to
determine the initial state of B,C and D as closed. I think this is the
root of the problem - we've put in something to override what used to be
the "show everything" section path (.-) and now we're looking at putting
in something to override the override. Going to get very messy.

If the sections were instead all peers:

[empty root section]
[stuff always displayed] (A)
[heading] (B)
[heading] (C)
[heading] (D)

Then we could have (A) as a static section (b="s"). That means it always
gets displayed open when its parent is open (ie always, as its parent
would be root). B, C and D would be normal sections - open when fed the
"-" section path, closed but heading visible when a section path is
supplied which doesn't apply to them.

T
- posted by Tim Bruce @ 11:58 AM 0 comments