pingest.pl Gets a New Option

In case you're using pingest.pl from MVLC's Evergreen utilities repository, then you might be interested to know that it got a new option this week:

--pipe
         Read record IDs to reingest from standard input.
         This option conflicts with --start-id and/or --end-id.


This new option allows you to run a custom query to feed record ids to pingest.pl. For instance, assuming you have a query that returns bibliographic record ids in a text file called query.sql, you could use a command line like the following to ingest the records corresponding to the ids returned from the query:

    psql -q -t -f query.sql | pingest.pl --pipe

In the absence of the --pipe option, pingest.pl continues to use its internal query to determine what records to ingest.

In case you are new here and don't know what all this record ingestion is about, this is Evergreen-speak for generating the indexes used for search, browse, facets, and record attributes. pingest.pl generates these indexes in parallel by splitting the records up into batches and working on more than one batch at a time. Parallel processing is usually faster than starting with one record and going straight through to the end.

pingest.pl Gets an Outside Contribution

Bill Erickson contributed a patch to pingest.pl that adds new command line options and cleans up the handling of the current options.

The command line options as of now are:

--batch-size

Number of records to process per batch

--max-child

Max number of worker processes

--skip-browse
--skip-attrs
--skip-search
--skip-facets

Skip the selected reingest component

--start-id

Start processing at this record ID.

--end-id

Stop processing when this record ID is reached

--max-duration

Stop processing after this many total seconds have passed.

--help

Show the help text and exit.

For those of you who may not know, pingest.pl is useful for reindexing records in your Evergreen database. It can reindex them in parallel thus reducing the time that it takes.

You can find pingest.pl as part of MVLC's Evergreen utilities git repository.

MVLC's Helper Functions Get an Update

If you have been using functions from MVLC's helperfuncs repo, you might be interested to know that they received an update today after nearly three-and-a-half years.

Today, I added checks for some of the optional parameters in the helper functions to create circ matrix and hold matrix matchpoints. Basically, in the cases where the functions have to look up an id in the database, they will raise an exception if that look up returns NULL.  This saves you from entering a matchpoint with a NULL value in one of these fields when you make a typo.

So, if you're using these functions, you might want to git pull and update them in your database.

More MARC Template Updates

MVLC has updated our MARC templates again.

We've fixed a typo in the 337$b of the map template. The value should have been "n" when it was "nc."

We also added a new template for Launchpad tablets.

MVLC catalogers will see the new template on Monday when we've restarted services after our move this weekend.

If you're from the community and you're interested, you can grab these templates for your own Evergreen installation via git. The branch is shared here and here. Everything is in the top commit on the branch, so if you're not running master, you can just cherry-pick the branch into yours. (You are installing from git, aren't you?)

Setting up the Evergreen Browser Staff Client on the Server

Yesterday, we installed the server components for the experimental Evergreen browser staff client on one of our virtual machines at MVLC. This blog post will take you through the steps involved in setting it up, provide pointers to the existing documentation, and explain a couple of gotchas or missing bits of documentation.

First, you need to follow all of the usual instructions for installing OpenSRF 2.4 and Evergreen.

Near the bottom of the OpenSRF installation instructions, you will find a section devoted to setting up the optional Websockets components. This section is not optional if you want to test drive the browser staff client. Pay careful attention to the note at the top of the section. The instructions all assume that you are running the commands as the root user.

If you follow the instructions exactly as written, you will encounter a problem on step 14.5. The assumption surrounding that cp command is that you have issued a cd command back to the directory where you have either extracted the OpenSRF code tarball or cloned the OpenSRF git repo. If you run the command as written without first returning to the location of the OpenSRF code, you will get an error about missing files.

On step 14.7, if you have altered your Evergreen's eg.conf file to indicate where your SSL certificates and chain files (if any) reside, then you will want to make the same changes to the apache2.conf file that gets installed in the previous step (14.6).

Note also, if you do this on Ubuntu 14.04 (Trusty Tahr), you will need to use the apache2ctl-websockets command to start and stop the Apache instance used for Websockets. For some reason, /etc/init.d/apache2-websockets does not work to control this Apache instance.

 The default configuration sets the second Apache instance to listen for Websockets connections on port 7680 and 7682. These ports will need to be opened in any firewall that you may have protecting the computer where Websockets is configured.

Now that the necessary OpenSRF configuration is complete, let's turn to setting things up for Evergreen. You will only need to follow these steps if you have checked out the Evergreen code via git. If you have downloaded a tarball from the 2.7 or 2.8 series of Evergreen, the steps should have already been done and the results included in the tarball.

You will find the current instructions for setting up the Evergreen side of the browser staff client in a file named Open-ILS/web/js/ui/default/staff/README.install in your local git clone. Benjamin Shum, the release manager of Evergreen 2.7 has summarized these instructions on the web. At some point in the near future, these instructions should be moved into the main installation instructions for Evergreen. My step references below come from Ben's compiled instructions.

If you follow those steps exactly, you will likely need to run chmod -R opensrf:opensrf ~/.npm/ between steps 4.1 and 4.2. When you do the sudo npm ... commands in steps 4.1.1, 4.1.2, and 4.1.3, npm will create a .npm settings directory in your home and it will be owned by root. You can avoid this by doing a sudo -i before running the three npm commands in 4.1. If you don't install as the opensrf user, but use some other user name, then you'll need to substitute that user's name in the chmod command.

If you are doing this setup after having already installed Evergreen, you will need to do the Evergreen make install again to put the files in the proper place. This means going back to the root of your git clone, issuing the sudo make install command, and then doing any chmod commands as necessary.

Finally, you'll want to run the following commands to remove some junk that you don't need hanging around:

cd /openils/var/web/js/ui/default/staff/
rm -rf bower_components
rm -rf node_modules 

Note:You can also delete those directories from the source before doing the make install step.

Once you have completed the above, and made sure that the Apache instance for Websockets is running, you should be able to access your browser staff client at https://your.server.domain/eg/staff/ .

If you have any trouble getting this to work, be sure to pop into the #evergreen channel on the Freenode IRC network. Someone may be able to help you depending on the time of day.

Feel free to add any comments or questions to this post. I can't guarantee timely answers, but I'll do my best.

Updated MARC Templates

MVLC has updated our MARC templates quite a bit over the past year. Highlights since our previous updates include:

• More RDA fields in most templates.
• Add copyright symbol to 264$c where 2nd indicator is 4 in book (ABK) and large print (ABKLP) templates.
• Add a video game template (VIDEOGAME). 

If you're interested, you can grab these templates for your own Evergreen installation via git. The branch is shared here and here. Everything is in the top commit on the branch, so if you're not running master, you can just cherry-pick the branch into yours. (You are installing from git, aren't you?)

Evergreen Update this Weekend

We have scheduled our next Evergreen update for the night of Sunday, December 8th, providing no deal-breaking issues are discovered with the next release between now and then.

Among the features you’ll see following the update are:

1. A mobile-friendly catalog
2. The addition of an author, title, subject and series Browse search mode
3. Larger fonts in the OPAC (by default) and labeled and repositioned search fields
4. The addition of a “My List Preferences” tab in Account Preferences where you can identify the number of lists and the number of list items that should display on each page. Individual lists will also display a page number and will be able to be navigated by page number
5. When pasting a barcode into a search filed, any extra white space will be automatically trimmed
6. Additional memory leaks tied to receipt printing are fixed. This should improve staff client performance
7. In circulation, lost items will now be included as part of the lump-sum tally of items out and will display with other checked out items, rather than below
8. The ability to print receipts for a single selected item is added from the interfaces for Items Out or Lost, Claims Returned, Long Overdue and Has Unpaid Billings

There’s still time to look at the new version before the update. We currently have it set up on the training server where you can put it through its paces.

If you currently have the staff client for the training server installed on your computer, you will be prompted to update your client when you attempt to log in.