Indexes still generating, but re-ran test on the semi-complete 1.4 native indexes:

Using lucene 1.2, 512 Mb allocated to Tomcat
w 50 5 http://www.test.xreferplus.com/results.jsp? term=|WORD| 1540 4796 642 1548 4809 644
w 50 5 http://www.test.xreferplus.com/results.jsp? term=|WORD| 1532 4027 700 1545 4029 702
w 50 5 http://www.test.xreferplus.com/results.jsp? term=|WORD| 1586 4509 659 1597 4537 661

Using lucene 1.4rc4, 512 Mb allocated to Tomcat

w 50 5 http://www.test.xreferplus.com/results.jsp? term=|WORD| 3337 14109 1203 3344 14112 1205

Using lucene 1.4rc4, 512 Mb allocated to Tomcat, indexes built using lucene 1.4rc4

w 50 5 http://www.test.xreferplus.com/results.jsp? term=|WORD| 811 3619 330 819 3622 332


which is more encouraging.

In the meantime, tried MultiSearcher as an alternative to the whole merged-indexes enterprise. Doesn't work: we get "Too many open files" errors. Tried adding -DdisableLuceneLocks=true to the command line, but we're still getting nowhere. Could try replacing FSDirectory with our own instrumented version which will tell us which files aren't getting closed, but that's a job for next week.

Waiting for remaining indexes so we can merge them and get a fair comparison on readingtest.

Glastonbury tomorrow.

T
- posted by Tim Bruce @ 4:15 PM