Feed API 2.0

From NUBWiki

Introduction

The Feed API v2.0 is a non-backwards compatible replacement of the existing v1.0 API. Perhaps the biggest change between the two versions is the underlying data model. In v1.0 there were three distinct constructs - productions, activities and locations. In v2.0, these have been merged into a single event construct. The advantage of this approach is that it is now possible to support a large set of search criteria in a single API.

The other large change between versions is the introduction of #Search Facets, which exploit the categorisation of Events to provide a powerful way for user's to refine their searches.

The following page will introduce these concepts in more depth as well as describing the API in full.

Events

The Feed API v2.0 is built around the concept of events. These are designed to closely match what we intuitively see an event as, something happening somewhere at sometime. While in v1.0 these three concepts where separated into productions, locations and activities respectively, v2.0 brings these into the single object - event. Everything that is known about that event, which includes everything known about when the event is occurring and the location its occurring at, is part of the event object. For those who are used to normalized data or relationship databases, this can seem a little counter-intuitive. If multiple events are occurring at the same location, then the location's details will be listed multiple times. Although this means the data is duplicated, the power of this de-normalization is that it is now possible to find events based on criteria related to whats happening, when its happening, and where its happening.

Nub XML

All responses from the Feed API v2.0 uses a XML format known as the Nub XML. The XSD can be found here. The following is a summary of the format:

 <nubxml>
   <suggestion>...</suggestion>
   <events start=... rows=... numfound=...>
     <event eventcidn=... productioncidn=... datecreated=... datechanged=...>
       ...
     </event>
   </events>
   <facets>
      <facet name=... param=...>
         ...
      </facet>
   </facets>
 </nubxml>

Contained in the XML are:

• #Spelling Suggestions
• Events found by the search
• #Search Facets

Searching for Events

Searching for events is done through HTTP GET requests to a REST interface. The following is a description of the interface.

Requests

Base URL: http://pscluster.uitburo.nl:8080/agenda/search.do

Parameters:

Note, events which have completed in the past cannot be requested and will not be included in the search results. Such requests will not result in an error, just an empty result list.

Examples

The following are examples of common searches:

• Find all events:

http://pscluster.uitburo.nl:8080/agenda/search.do?key=39a8629319cc746839f929a1444b2598

• Find all events involving Hamlet:

http://pscluster.uitburo.nl:8080/agenda/search.do?key=39a8629319cc746839f929a1444b2598&text=hamlet

• Find all Dans events:

http://pscluster.uitburo.nl:8080/agenda/search.do?key=39a8629319cc746839f929a1444b2598&headGenre=Dans

• Find all Dans events happening at Het Muziektheater Amsterdam:

http://pscluster.uitburo.nl:8080/agenda/search.do?key=39a8629319cc746839f929a1444b2598&headGenre=Dans&location=Het%20Muziektheater%20Amsterdam

• Find all events sorted by startdatetime:

http://pscluster.uitburo.nl:8080/agenda/search.do?key=39a8629319cc746839f929a1444b2598&sort=startdatetime&direction=asc

• Find all events in Amsterdam Uitburo region:

http://pscluster.uitburo.nl:8080/agenda/search.do?key=39a8629319cc746839f929a1444b2598&regionId=1

Locales

One major difference between the Feed API 2.0 and 1.0 is the support for multiple locales. Through the locale parameter it is possible to search for events in a specific locale. For simplicity, searches are only done on events which have a translation in the specified locale. Therefore although an event might be found when searching in nl_NL, it might not be found when searching in fy_NL. The following table defines the locales and their parameter values:

Regions

Each event in the Feed API is assigned a specific region id, which connects an event to its owning Uitburo. The following table defines the regions with their region ids:

Dates & Times

The Feed API v2.0 uses UTC for all its datetimes, in the following format: The Feed API v2.0 uses datetimes in the following format: yyyy-MM-dd'T'HH:mm:ss'Z'. See here for a description of the format.

The only exception to this is the <starttime/> element which is stored as text and is in CET time.

Some events are entered into the system without a time in their startdatetime. Such events are given the start time 06:03 CET (05:03 UTC). Equally, some events do not have enddatetimes. These events are given given the startdatetime as their enddatetime.

Periods

Periods are simple shorthand values for common date periods. The following table defines the current periods with their full date ranges:

Periods can be used in search requests through the period parameter.

Relevancy

At the heart of the Feed API 2.0 is the concept of search result relevancy. As opposed to a traditional relational database based system, where the order of how results are returned to the user is less important, free text search system such as the Feed API 2.0 are driven by the concept of a relevancy score. When a search is executed, a relevancy score is computed for each search result. The score is computed using Apache Lucene's academically proven algorithm which is described here.

Part of the scoring algorithm is field specific boosting. This allows matches on specific fields in events to be considered more important than matches on other fields. The order of importance defined in the Feed API 2.0 is as follows:

1. title
2. location name
3. gezelschappen
4. medewerker name
5. head genre
6. sub genre
7. summary, short description, description

Matches on combinations of these fields will be combined together when computing the final score. Therefore a match on title and head genre will be more important than just a match on title.

Search Facets

In addition to events, the Feed API 2.0 includes search facets. Conceptually, search facets use the categorisation of events in a search result list, to provide ways to filter the list further.

Imagine a situation where a user searches for 'Anouk' using the text parameter defined above. This will result in a wide array of events occurring at different locations. Through the location search facet, it would be possible to see how many of these events are occurring at each location. Using the location values, it would then be possible to filter the search results to see only those events happening at Amsterdam Arena, for example.

The facet information included in the Feed API 2.0 results follows the following format:

 <facet name="FACET_NAME" param="FACET_PARAM">
   <facetEntry count="COUNT">VALUE</facetEntry>
   ...
 </facet>

Here:

• FACET_NAME is a logical name for the facet
• FACET_PARAM defines the parameter to use in combination with VALUE to filter the search results
• COUNT defines the number of events in the total search results (not just the current page) that have the value VALUE
• VALUE is the value that the events have for the facet

Therefore assuming the following example:

 <facet name="HeadGenre" param="headGenre">
   <facetEntry count="240">Cabaret</facetEntry>
   <facetEntry count="384">Dans</facetEntry>
   <facetEntry count="38">Film</facetEntry>
 </facet>

For the facet 'HeadGenre' there exists 240 events with the value 'Cabaret', 384 with 'Dans' and 38 with 'Film'. To then filter the results to those with 'Film', you would append headGenre=Film to your request.

The Feed API 2.0 defines 4 facets:

1. HeadGenre (based on the event head genres)
2. SubGenre (based on the event sub genres)
3. Location (based on the event location titles)
4. Period

Spelling Suggestions

Another component of the response from the Feed API 2.0, is spelling suggestions. Included at the top of the NUBXML under the <suggestion/> tag, spelling suggestions are suggestions for new text parameter values, based on those provided in a request. Internally, the text parameter value is compared against indexed terms and a suggestion that will result in the highest number of results is generated.

Consider a search with text=hamlek. Such a search will most likely result in 0 results. In this situation, the Feed will suggest the correctly spelled 'hamlet' as an alternative text value. The following is the XML illustrating this example:

 <nubxml>
   <suggestion>hamlet</suggestion>
   <events start="0" rows="10" numfound="0">
   </events>
   ...
 </nubxml>

Note, suggestions are segregated on a per region basis. This means that when searching with the regionId=1, only those events with this region id will be used as a source of suggestions. Also, when the generated suggestion is the same as the provided text parameter value, the suggestion will not be included in the search response.