ILL Module

== Chat Log (30 Oct 2015) ==
<atheia> So I spent about two week working on an abstract interface to ILL.
<magnuse> cool
<atheia> The idea is, as described in the RFC, etc., to have a unified
 frontend / koha-module [13:18]
<atheia> And then a number of backends that implement the interaction to the
 specific ILL system in use.
<atheia> So for instance, I saw that you had written some custom .pls for the
 user interface.
<atheia> Instead the idea would be that even with such diverging APIs as NNCIP
 and BLDSS, we could still use the same front-end scripts. [13:19]
<magnuse> yeah, mostly because i wanted to experiment in my own little sandbox
 :-)
<atheia> Originally the idea was to achieve this using a well-defined
 interface, and then some way to pass "form snippets" back from the
 API backend to the template. [13:20]
<atheia> But this breaks down for several reasons — one of which is
 internationalization — another being complexity.
<magnuse> yeah...
<atheia> So what I'm settling on now is to have: a) a clean facade of ILL
 methods, used from the Koha side [13:21]
<atheia> b) each method in the facade can take an additional "template"
 parameter, by which the API backend can perform additional steps for
 that part of the API logic [13:22]
<atheia> c) the front-end templates use additional includes (one set of
 includes for each API backend), which also get passed the "template"
 parameter, so those include snippets can generate forms as necessary
 for that particular part of the API backend logic. [13:23]
<atheia> I don't know if this is now utterly confusing — I could try to
 illustrate? alternatively if you have question?
<atheia> I'll point you to the relevant code repo though. sec.
<magnuse> hehe, i can't say i have a clear mental image of it yet... [13:24]
<magnuse> but it does sound flexible
<atheia> Right :-) [13:25]
<atheia> The branch I've been committing to for this is:
 https://github.com/PTFS-Europe/koha/tree/ill_community
<atheia> But as an example:
<magnuse> the way i am looking at it now, nncipp is mainly about changing
 statuses and having those status changes fire off messages to the
 other library. and maybe have the responses from those messages
 change the statuses again [13:26]
<atheia> OK.
<magnuse> ah yes, lots of stuff going on in that branch i see [13:27]
<atheia> And when you say change statuses — are these the ILL statuses that
 are now an authorized list?
<magnuse> yup
<atheia> And how do you create a "new ill request"?
<magnuse> but that might not be relevant to what you were about to say :-)
 [13:28]
<atheia> Do you manually fill in a form or, search against the stock of
 partner libraries?
<atheia> No, it kind of is, I think :-)
<magnuse> for now, i fill out a form
<magnuse> but i can totally see a z39.50/sru search being the starting point
<atheia> Right; we currently have two approaches too: form and API websearch.
 [13:29]
<magnuse> also, we have this thing where any library can log into the opac of
 any other library and place a request there, which is then sent back
 to the home ils
<atheia> oh
<atheia> wow. so that's a totally different workflow.
<atheia> Would that still use nncipp or would that be something different
 altogether?
<atheia> Like, would that create an ill record in the requesting library's ILL
 module? [13:30]
<magnuse> well, the way it will be implemented is: the opac sends a message to
 the home ils, then the home ils sends a message to the owner ils
 (which is the same as the message sent when you fill out a form)
<magnuse> sorry, phone
<atheia> np
<magnuse> gah [13:31]
<magnuse> i need to pick up my wife
<magnuse> might be gone about 20 minutes
<atheia> :-) np — we can pick it up later again. Already food for thought.
<atheia> Just ping me in the private chate.
<atheia> s/chate/chat/
<magnuse> ok, great! 
<atheia> later!
<magnuse> back! [14:11]
<atheia> Ah sorry — didn't see your message. [14:34]
<atheia> you still about?
<magnuse> yup
<magnuse> i should have shouted louder ;-) [14:35]
<atheia> :-)
<atheia> k
<atheia> So — if I understand correctly, the workflow you are using is:
 [14:36]
<atheia> - create request (currently form, could be z39 search, or OPAC
 message)
<atheia> - update request status to trigger further messages [14:37]
<magnuse> yup [14:38]
<magnuse> oh and there are also incoming messages that result in a changed
 status. i use the NCIPServer rangi has been working on for that
 [14:39]
<atheia> And would these incoming messages, which cause a status change, then
 in return cause further "business logic" to be triggered? [14:40]
<magnuse> so i order something and it gets status ORDERED. then the owner
 library sends it, and send a message at the same time, so my request
 is upgraded to SHIPPING. then i receive it and change the status to
 RECEIVED, and send a message to the owner library that it has been
 recieved
<magnuse> hm, i think they are mostly informative [14:41]
<atheia> Aha.
<magnuse> there is renewal [14:42]
<magnuse> i can send off a request for renewal and the answer can be yes or no
<atheia> Right — but that is more of an "action" rather than a status update
 right?
<magnuse> yeah, probably
<atheia> BLDSS API also has a "status" action whereby we can request
 information on what's going on with a request (e.g. see: "order
 received" or "dispatched"). [14:43]
<atheia> So the feeling I'm getting here is that you could do something like
 "status update hooks":
<atheia> whenever the "status update" action is performed, we also make a
 method call to the API backend passing the STATUSCODE [14:44]
<atheia> The API can then perform actions (like send a message to a remote
 server), and returns OK
<atheia> After which ILL updates the status.
<atheia> That would handle all your status update logic no?
<magnuse> yup, sounds like it [14:45]
<atheia> OK — that is doable… I reckon we could use the same model as what I'm
 doing now (he said optimistically) [14:46]
<atheia> Whereby the backend can return a "partial-success" with a specific
 TEMPLATE, which in turn is passed back to the TT template, so it can
 include an INCLUDE which can display a specific form for that
 TEMPLATE.
<atheia> But — would the status updates require any further manual
 intervention, or is it basically automatic after that? [14:47]
<atheia> i.e.: when you update to RECEIVED, do you need to pass additional
 data to the backend for their call to a remote server (e.g. the
 recipients comments).
<magnuse> as far as i can tell it's automatic [14:48]
<atheia> OK
<atheia> Then ignore my elaborations — let's KISS
<atheia> I… mean… you know.
<atheia> keep it simple.
<atheia> *blush*
<magnuse> no, i see changing the status to received as just reading the
 barcode off the book, and then koha should shoot a "received"
 message to the owner library
<magnuse> lol! [14:49]
<magnuse> yeah, let's keep it simple :-)
<atheia> :-) good
<atheia> Ah.
<atheia> But we need to pass to read the barcode and pass that to the backend
 right?
<atheia> I imagine that would not necessarily be the same as the "request id"
 — i.e. would the information already be contained in the request data
 stored in the db? [14:50]
<atheia> So: workflow detail.
<atheia> please. :-)
<magnuse> the way i do it now, i read the barcode and use that to look up the
 request data in the db [14:51]
<magnuse> over time there might be multiple requests for the same barcode, so
 i use barcode and status to do the actual lookup [14:52]
<atheia> is th ebarcode the item barcode or borrower barcode?
<magnuse> when i want to receive (and send off RECEIVED messages) there should
 only be one record with the barcode and the status SHIPPED
<magnuse> item barcode [14:53]
<atheia> of course.
<atheia> ok.
<atheia> So you barcode search an item, get to the ILL in the ILLrequest
 module
<atheia> Then set status to the RECEIVED status
<atheia> right? [14:54]
<magnuse> yup
<atheia> ok, in which case, I think simple works.
<magnuse> or what i actually imagine is a form for reading barcodes and
 setting status RECEIVED/sending RECEIVED messages [14:55]
<magnuse> so you'd open all the parcels you got that day
<magnuse> pile up all the ill books and scan them
<magnuse> into a text field [14:56]
<atheia> ok, yeah, I think that works.
<magnuse> and one thing i have not thought about so far: that should trigger a
 message to the borrower that his/her book has arrived [14:57]
<magnuse> and probably also a slip to put in the book before it is placed on a
 shelf to be picked up
<atheia> right — for now, I would suggest we kick that in the long grass.
<atheia> Just to keep the whole thing manageable.
<magnuse> yup [14:58]
<magnuse> that is what i have been doing :-) 
<atheia> I could see us adding these features either to the status change bit
 or in the API backend…
<atheia> dunno.
<atheia> good call :-)
<magnuse> (might be able to piggyback on existing holds functionality)
<atheia> Right — I think that's what Janet over here has been doing: [14:59]
<atheia> She added a UI button that allows users to create a Koha item from an
 ILLRequest, which is then added to the usual holds for the borrower.
<atheia> *I think*
<atheia> So do you feel like you want to add more context? [15:00]
<magnuse> yeah, sounds reasonable
<atheia> Or can I try to frame this in terms of the work I've been doing?
<magnuse> hm, not that i can think of
<atheia> So we can see how feasible it is for you?
<magnuse> sure
<atheia> OK.
<atheia> So I would propose 2 points of contact between the Koha ILL module
 and a specific backend: [15:01]
<atheia> 1) actions (these include things like "creating a new ILLRequest", or
 "Renewing" or "Fetch Status";
<atheia> 2) status update hooks (these would be things like, set status to
 received, send message to remote host; or set status to COMPLETE, add
 data to the COMPLETION_DATE field) [15:02]
<atheia> For 1): the UI would expose specific buttons for each available
 action in the "manage_ill" interface. [15:03]
<magnuse> sounds good to me
<atheia> When the user clicks on the appropriate button, the relevant method
 is called in the API backend, which responds either by performing
 action automatically (e.g. firing off a renewal request and reporting
 success) or by responding with the name of a TEMPLATE.
<atheia> In the latter case, the ILL UI then includes that Backend's Method
 INCLUDE, passing the TEMPLATE name, and that include can present a
 form to the end user where they can provide additional data. [15:04]
<atheia> For instance, for BL API: when placing the order, the API backend
 passes back for the 'confirm' TEMPLATE, which causes the UI to
 generate a form with a selection of delivery services/prices. [15:05]
<atheia> The user selects the price and clicks submit.
<atheia> At this point the UI, knowing that it invoked a TEMPLATE, calls the
 same method in the backend again, with the new data from the user,
 and the backend then performs the operation [15:06]
<atheia> In our example — the backend then uses the price information to
 finally place the order with BL.
<atheia> So we have something like 4 different actions (create order, confirm
 order, get status, renew loan); but each Backend is able to perform
 as many intermediary steps as required to complete each of those
 actions. [15:07]
<atheia> Conversely, in your case, your backend might simply not implement
 "confirm", or "status"
<atheia> as they don't make sense in your context. [15:08]
<magnuse> yeah
<atheia> Does this all make sense so far?
<magnuse> i think so, yes :-)
<atheia> OK.
<atheia> So for 2)
<atheia> The UI offers, in the ill-manage part, a chance for an end-user to
 select a new status for an ILL. [15:09]
<atheia> Upon submission, the UI invokes, always the same method:
 UPDATE_STATUS (or something), and passes the new status code.
<atheia> The backend upon which that method is invoked must handle each
 possible status code passed to it. [15:10]
<atheia> (but it can choose to not perform any action for a specific code.
<atheia> )
<magnuse> (and there should be some limit on "status paths", so from status A
 you can only move to status B or C, not D)
<atheia> At this stage, the Backend can perform a specific action (like
 sending a message to a remote server).
<atheia> OK. Interesting! [15:11]
<atheia> Do you feel like that limitation would have to be present on the UI
 (e.g. only show those statuses that are allowed in this context)
<magnuse> well, it would be nice
<atheia> or would it be ok to always show all statuses, but let the backend
 say "tihs makes no sense"
<atheia> Indeed :-) I also think it would be more complex for now. [15:12]
<magnuse> yeah
<atheia> We'd need to track status dependencies somehow.
<atheia> If possible, I'd prefer the second (where the backend returns "this
 makes no sense")
<atheia> WDYT?
<magnuse> as a first pass i think it would be ok to rely on librarians to know
 which changes makes sense
<atheia> awesome.
<atheia> :-)
<atheia> So in order to implement this, I think the "UPDATE_STATUS method
 should take two arguments: new_status_code, old_status_code. [15:13]
<atheia> Then the backend can evaluate whether this is a valid transition, and
 perform it's actions if it is.
<atheia> WDYT?
<magnuse> works for me :-)
<atheia> cool.
<atheia> So the backend performs its action and returns "success" or "fail".
 [15:14]
<atheia> If "success" the Koha ILL Module finally updates the status in the
 ILL tables to the new status.
<atheia> If "fail" the Koha ILL Module displays an error message and does not
 update the status in the ILL tables.
<atheia> And I think… that's all :-)
<magnuse> hehe, yeah [15:15]
<atheia> Now most importantly, do you think that this kind of infrastructure
 would work for you as a backend implementor?
<atheia> (you know, with help and documentation :-) )
<magnuse> what i have done in a couple places is change the status to
 RENEWAL_REQUESTED before the renewal request is ssent, and then
 RENEWAL_OK after i get the confirmation. that way i can pick up
 failed requests [15:16]
<magnuse> yeah, it sounds kind of logical :-)
<atheia> OK, so that is a slightly more manual approach to the renewal
 process.
<atheia> is renewal a realtime request for you in NNCIPP or is it
 asynchroneous? [15:17]
<magnuse> with clear or it's just me being lazy and not taking all possible
 outcomes into account...
<atheia> i.e. does the remote service provide an immediate response or do they
 manually process requests?
<atheia> Sorry — I don't think I understand your last sentence? [15:18]
<magnuse> s/with clear//
<atheia> ah 
<atheia> ;-)
<magnuse> immediate response
<atheia> k, so you could implement that as a real-time action, as discussed
 above and save yourself the hassle of the 2 additional ILL statuses.
<magnuse> yeah
<atheia> But to clarify, as an implementor, you would basically implement:
 [15:19]
<atheia> a) a Backend, which implements the 4 actions as methods as mentioned
 above, and 
<atheia> b) a method for the backend called UPDATE_STATUS
<magnuse> yeah, that makes sense [15:20]
<atheia> c) implement includes for the ILL templates for your specific
 Backend, which would be either stubs (in case your backend has no
 multi stage ACTIONS), or which implement additional forms.
<atheia> And that's all there should be to it for an implmentor.
<atheia> cool
<atheia> Unfortunately I am not quite there yet — but I am implementing this
 system now on ill_community. [15:21]
<magnuse> shpuld be doable in an afternoon ;-) 
<atheia> optimistic ;-)
<magnuse> hehe
<magnuse> do you have an idea about a timeframe for that [15:22]
<magnuse> ?
<magnuse> i have my work based on the ill_master branch, i think. i need to
 polish that to where it cooperates with the other norwegian
 implementations (so i can get some of my money from the project :-)
 [15:23]
<atheia> Yeah, I reckon I need another 2 weeks to get the infrastructure ready
 and to implement the ACTIONS and UPDATE_STATUS, and template includes
 for BLDSS API.
<atheia> OK.
<magnuse> the next stage of the project is to get things into production
<atheia> What I can do is imlement the interface in the abstract (i.e. without
 the BLDSS backend).
<atheia> and then let you know, and you could implement NNCIPP at the same
 time that I implement the rest of BLDSSAPI. [15:24]
<magnuse> that sounds like a perfect fit, if i can take the ncip/backend code
 i have now and start to bake it into your framework
<atheia> and that way we can chat and improve things in the abstract
 interfaces whilst we go along?
<atheia> OK.
<magnuse> sounds like a really awesome plan
<atheia> I will ask Martin if I can get another day next week to do that
 specific thing, and then can let you know when you can start using
 ill_community. [15:25]
<atheia> sec.
<magnuse> great
<atheia> OK. [15:26]
<atheia> That's good.
<atheia> I'll make sure to be in IRC everyday I'm actually working (I'll be
 off on Wednesday/Thursday). [15:27]
<magnuse> ok
<atheia> And plan to ping you with the green light on Tuesday at the latest.
<magnuse> awesome
<atheia> So then we can continue chatting as we go along.
<atheia> yeah deffo!
<magnuse> don't rush it for my sake, though
<atheia> It's really helpful to have you work on this whilst we work on it
 too. [15:28]
<magnuse> i need to fix an encoding thing in the code i have
<magnuse> then get a "signoff" from another vendor, that our systems can talk
 together
<magnuse> then i'll probably have some other stuff to attend to
<atheia> OK, well, I'll get this part done anyway, and then we can revisit it
 whenever you have another moment to look at this.
<magnuse> so it will be a week or two before i can start looking at the
 "remake"
<atheia> :-)
<magnuse> great
<atheia> np
<atheia> Hey, another question, [15:29]
<atheia> Would you mind if I update the RFC/ Bug with a log of this chat?
<atheia> So we can kind of share progress etc. without me having to distill
 everything immediately?
<atheia> (np if you'd rather not)
<magnuse> please do [15:30]
<magnuse> i don't mind [15:31]
<atheia> awesome :-)
Category:Circulation_RFCs