InsightUBC (2.0.3)

Deprecated. Use /api/v2/datasets.

Upload a zip archive for asynchronous processing. The data is normalized and added to existing resources in the system.

Uploading Course Offerings

To bulk upload course offerings (combined course and section data flattened into a single record), make a multipart request with:

1. The kind field set to "course_offerings".
2. The archive field set to a zip file with:

   • a root directory named courses/

   • one or more JSON files inside courses/ (file names are arbitrary)

   • each JSON file has a result property whose value is an array of objects with the following partial schema of fields that must be present together with their expected type:

     {
       id: string,
       Course: string,
       Title: string,
       Professor: string,
       Subject: string,
       Section: string,
       Year: string,
       Avg: number,
       Pass: number,
       Fail: number,
       Audit: number
     }
     

Once the upload is complete, the system asynchronously processes the uploaded archive. The upload job is tracked with a unique id that you can use to query the job status via the GET /api/v1/datasets/{id} endpoint. The processing follows these steps:

1. Validate zip format: The uploaded file is verified to be a valid zip archive. If not, the job status is set to failed with message "Data is not in a valid zip format".

2. Check for courses/ directory: The system checks for the existence of a root courses/ directory within the archive. If missing, the job status is set to failed with message "Missing root courses directory".

3. Process files in courses/ directory: For each file in the courses/ directory:

   • If the file is valid JSON with a result property, each course offering object in the result array is processed according to the rules below.
   • If the file is not valid JSON or does not have a result property, the file is skipped.

4. Load records into resources: For each result object that is processed, the following rules apply:

   Courses: the course id is set to Subject + Course (the concatenation of the field values).

   • If the course does not exist, create a new course with:

     • code set to the offering's Course field
     • title set to the most recent offering's Title field
     • dept set to the offering's Subject field

   • If the course exists, update it only if any field value in the mapping above differs from the current resource.

   Sections: the section id is set to the course offering id.

   • If the section does not exist, create a new section under the course with:

     • instructor set to the offering's Professor field
     • year set to the offering's Year field, converted to a number (or 1900 when the offering's Section field equals "overall")
     • avg set to the offering's Avg field
     • pass set to the offering's Pass field
     • fail set to the offering's Fail field
     • audit set to the offering's Audit field

   • If the section exists, update it only if any field value in the mapping above differs from the current resource.

   A record will not be processed if any of the following conditions are met:

   • A required field listed in the above schema is missing
   • A required field's value cannot be converted to the expected type

   Note: Courses are processed before sections to ensure every section's parent course exists before the section is created or updated.

Deprecated. Use /api/v2/datasets/{id}.

Retrieve the status and statistics of a dataset upload job. The status reflects the current processing state:

• processing: The job is currently being processed according to the steps defined in the POST endpoint.
• completed: The job finished successfully. File statistics show how many files were processed vs skipped. Resource statistics show how many resources were created or modified during processing.
• failed: The job encountered a fatal error during validation. Check the message field for details (e.g., "Data is not in a valid zip format" or "Missing root courses directory").

Upload a zip archive for asynchronous processing. The data is normalized and added to existing resources in the system.

Important: This endpoint returns a 202 response immediately upon accepting the upload. All validation and processing steps described below happen asynchronously in the background. To check the status of processing or see any errors that occurred, query the GET /api/v2/datasets/{id} endpoint using the returned job id.

Uploading Course Offerings

To bulk upload course offerings (combined course and section data flattened into a single record), make a multipart request with:

1. The kind field set to "course_offerings".
2. The archive field set to a zip file with:

   • a root directory named courses/

   • one or more JSON files inside courses/ (file names are arbitrary)

   • each JSON file has a result property whose value is an array of objects with the following partial schema of fields that must be present together with their expected type:

     {
       id: string,
       Course: string,
       Title: string,
       Professor: string,
       Subject: string,
       Section: string,
       Year: string,
       Avg: number,
       Pass: number,
       Fail: number,
       Audit: number
     }
     

Course Offerings Processing Steps:

The following steps occur asynchronously after the upload is accepted. Check the GET /api/v2/datasets/{id} endpoint to monitor progress and see any errors.

1. Validate zip format: The uploaded file is verified to be a valid zip archive. If not, the job status is set to failed with message "Data is not in a valid zip format".

2. Check for courses/ directory: The system checks for the existence of a root courses/ directory within the archive. If missing, the job status is set to failed with message "Missing root courses directory".

3. Process files in courses/ directory: For each file in the courses/ directory:

   • If the file is valid JSON with a result property, each course offering object in the result array is processed according to the rules below.
   • If the file is not valid JSON or does not have a result property, the file is skipped.

4. Load records into resources: For each result object that is processed, the following rules apply:

   Courses: the course id is set to Subject + Course (the concatenation of the field values).

   • If the course does not exist, create a new course with:

     • code set to the offering's Course field
     • title set to the most recent offering's Title field
     • dept set to the offering's Subject field

   • If the course exists, update it only if any field value in the mapping above differs from the current resource.

   Sections: the section id is set to the course offering id.

   • If the section does not exist, create a new section under the course with:

     • instructor set to the offering's Professor field
     • year set to the offering's Year field, converted to a number (or 1900 when the offering's Section field equals "overall")
     • avg set to the offering's Avg field
     • pass set to the offering's Pass field
     • fail set to the offering's Fail field
     • audit set to the offering's Audit field

   • If the section exists, update it only if any field value in the mapping above differs from the current resource.

   A record will not be processed if any of the following conditions are met:

   • A required field listed in the above schema is missing
   • A required field's value cannot be converted to the expected type

   Note: Courses are processed before sections to ensure every section's parent course exists before the section is created or updated.

Uploading Facilities

To bulk upload facilities (rooms and buildings), make a multipart request with:

1. The kind field set to "facilities".
2. The archive field set to a zip file with:
   • an index.htm file at the root of the zip
   • one or more HTML files containing building and room information linked from index.htm

Facilities Processing Steps:

The following steps occur asynchronously after the upload is accepted. Check the GET /api/v2/datasets/{id} endpoint to monitor progress and see any errors.

1. Validate zip format: The uploaded file is verified to be a valid zip archive. If not, the job status is set to failed with message "Data is not in a valid zip format".

2. Validate index.htm: The system checks for the existence of index.htm at the root of the archive. If missing, the job status is set to failed with message "Missing index.htm file". The file is then parsed to verify it is valid HTML. If it cannot be parsed, the job status is set to failed with message "index.htm could not be parsed".

3. Locate building table: A table with class views-table is located in the parsed index.htm. If no such table is found, the job status is set to failed with message "No building table found in index.htm".

4. Extract building information: For each row in the building table, the following information is extracted:

   • fullname: Text content from the <a> element in the cell with class views-field-title
   • shortname: Text content from the cell with class views-field-field-building-code
   • address: Text content from the cell with class views-field-field-building-address
   • link: The href attribute from the <a> element in the cell with class views-field-title
   • A row is skipped if any required element with the specified CSS class is not found.

5. Extract room information: For each link identified in index.htm:

   • The linked file is read relative to the zip root
   • A table with class views-table is located in the linked file
   • For each row in the table, the following information is extracted:
     • number: Text content from the <a> element in the cell with class views-field-field-room-number
     • seats: Text content from the cell with class views-field-field-room-capacity
     • furniture: Text content from the cell with class views-field-field-room-furniture
     • type: Text content from the cell with class views-field-field-room-type
     • href: The href attribute from the <a> element in the cell with class views-field-nothing
     • A row is skipped if any required element with the specified CSS class is not found.
   • If the file does not exist in the zip, the file cannot be parsed, or the room table is not found in the file, no room information is extracted for that building (but the building is still valid)

6. Extract geolocation information: For each building extracted in step 4:

   • A GET request is sent to the geolocation service using the URL-encoded building address extracted from index.htm
   • The response provides:
     • lat: The latitude value returned by the geolocation service
     • lon: The longitude value returned by the geolocation service
   • If the request fails or the response contains error, no geolocation information is available for that building and the building should be skipped.

7. Load records into resources: For each building and its rooms that were successfully processed in steps 4-6, the following rules apply:

   Buildings: the building id is set to shortname.

   • If the building does not exist, create a new building with:

     • name set to the building's fullname
     • address set to the building's address
     • lat set to the latitude from geolocation
     • lon set to the longitude from geolocation

   • If the building exists, update it only if any field value in the mapping above differs from the current resource.

   • The resulting building resource is validated against the same requirements as building data added through PUT /api/v2/buildings/{building}. If required building data is missing or invalid, no building resource is loaded.

   Rooms: the room id is set to shortname_number (the concatenation of building shortname, underscore, and room number).

   • If the room does not exist, create a new room under the building with:

     • building set to the building's shortname
     • number set to the room's number field
     • seats set to the room's seats field
     • type set to the room's type field
     • furniture set to the room's furniture field
     • href set to the room's href field

   • If the room exists, update it only if any field value in the mapping above differs from the current resource.

   • The resulting room resource is validated against the same requirements as room data added through PUT /api/v2/buildings/{building}/rooms/{room}. If required room data is missing or invalid, no room resource is loaded.

   Note: Buildings are processed before rooms to ensure every room's parent building exists before the room is created or updated.

Retrieve the status and statistics of a dataset upload job. The status reflects the current processing state:

• processing: The job is currently being processed according to the steps defined in the POST endpoint.
• completed: The job finished successfully. Statistics are kind-specific and report resource creation/modification outcomes during processing.
• failed: The job encountered a fatal error during validation. Check the message field for details (e.g., "Data is not in a valid zip format" or "Missing root courses directory" or "Missing root rooms directory").

Deprecated. Use /api/v2/search.

Execute a query using a JSON-based domain-specific language (DSL) to search resources.

The query object has three main components:

• WHERE filters which records should be included in the results.
• COLUMNS selects which keys are included in each result.
• ORDER sorts the results by a key in COLUMNS.

Query Grammar

The DSL grammar is defined using EBNF:

QUERY ::='{' BODY ', ' OPTIONS '}'

// Note: a BODY with no FILTER (i.e. WHERE:{}) matches all entries.
BODY ::= 'WHERE:{' FILTER? '}'

FILTER ::= LOGICCOMPARISON | MCOMPARISON | SCOMPARISON | NEGATION

LOGICCOMPARISON ::= LOGIC ':[' FILTER_LIST ']'
MCOMPARISON ::= MCOMPARATOR ':{' mkey ':' number '}'
SCOMPARISON ::= 'IS:{' skey ': "' [*]? inputstring [*]? '" }'  // Asterisks at the beginning or end of the inputstring should act as wildcards.
NEGATION ::= 'NOT :{' FILTER '}'

FILTER_LIST ::= '{' FILTER '}' | '{' FILTER '}, ' FILTER_LIST // comma separated list of filters containing at least one filter
LOGIC ::= 'AND' | 'OR'
MCOMPARATOR ::= 'LT' | 'GT' | 'EQ'

OPTIONS ::= 'OPTIONS:{' COLUMNS '}' | 'OPTIONS:{' COLUMNS ', ORDER:' key '}'

COLUMNS ::= 'COLUMNS:[' KEY_LIST ']'

KEY_LIST ::= key | key ', ' KEY_LIST // comma separated list of keys containing at least one key

key ::= mkey | skey
mkey ::= '"' mfield '"'
skey ::= '"' sfield '"'
mfield ::= 'avg' | 'pass' | 'fail' | 'audit' | 'year'
sfield ::=  'title' | 'dept' | 'code' | 'instructor'
inputstring ::= [^*]* // Zero or more of any character, except asterisk.

Wildcards

You can use optional asterisks (*) in IS comparisons to match patterns. For example, "IS": {"dept": "C*"} matches any department that starts with "C". Because the asterisks can appear at the beginning and/or end, there are four possible forms:

inputstring: Matches inputstring exactly
*inputstring: Ends with inputstring
inputstring*: Starts with inputstring
*inputstring*: Contains inputstring

Asterisks in the middle are not allowed (for example, input*string).

Order

You can sort results by specifying the ORDER field with any key that appears in your COLUMNS list.

If ORDER is not specified, results may appear in any order.

Ties. When sorting produces tied values, the relative order of those entries is not guaranteed. For example, if you sort by department, all CPSC entries will be grouped together, but individual CPSC entries may appear in any order within that group.

Execute a query using a JSON-based domain-specific language (DSL) to search resources.

The query object has three main components:

• WHERE filters which records should be included in the results.
• COLUMNS selects which keys are included in each result.
• ORDER sorts the results by a key in COLUMNS.

Important: All fields used in a query must belong to the same kind. You cannot mix course_offerings fields with facilities fields in a single query.

Query Grammar

The DSL grammar is defined using EBNF:

QUERY ::='{' BODY ', ' OPTIONS '}' | '{' BODY ', ' OPTIONS ', ' TRANSFORMATIONS '}'

// Note: a BODY with no FILTER (i.e. WHERE:{}) matches all entries.
BODY ::= 'WHERE:{' FILTER? '}'

FILTER ::= LOGICCOMPARISON | MCOMPARISON | SCOMPARISON | NEGATION

LOGICCOMPARISON ::= LOGIC ':[' FILTER_LIST ']'
MCOMPARISON ::= MCOMPARATOR ':{' mkey ':' number '}'
SCOMPARISON ::= 'IS:{' skey ': "' [*]? inputstring [*]? '" }'  // Asterisks at the beginning or end of the inputstring should act as wildcards.
NEGATION ::= 'NOT :{' FILTER '}'

FILTER_LIST ::= '{' FILTER '}' | '{' FILTER '}, ' FILTER_LIST // comma separated list of filters containing at least one filter
LOGIC ::= 'AND' | 'OR'
MCOMPARATOR ::= 'LT' | 'GT' | 'EQ'

OPTIONS ::= 'OPTIONS:{' COLUMNS '}' | 'OPTIONS:{' COLUMNS ', ' SORT '}'

SORT ::= 'ORDER: { dir:'  DIRECTION ', keys: [ ' ANYKEY_LIST '] }' | 'ORDER: ' ANYKEY
DIRECTION ::= 'UP' | 'DOWN'

TRANSFORMATIONS ::= 'TRANSFORMATIONS: {' GROUP ', ' APPLY '}'
GROUP ::= 'GROUP: [' KEY_LIST ']'
APPLY ::= 'APPLY: [' APPLYRULE_LIST? ']'
APPLYRULE_LIST ::=  APPLYRULE | APPLYRULE ', ' APPLYRULE_LIST
APPLYRULE ::= '{' applykey ': {' APPLYTOKEN ':' KEY '} }'
APPLYTOKEN ::= 'MAX' | 'MIN' | 'AVG' | 'COUNT' | 'SUM'

COLUMNS ::= 'COLUMNS:[' ANYKEY_LIST ']'

// Comma-separated list of keys containing at least one key
KEY_LIST ::= KEY | KEY ', ' KEY_LIST
ANYKEY_LIST ::= ANYKEY | ANYKEY ', ' ANYKEY_LIST

ANYKEY ::= KEY | applykey
KEY ::= mkey | skey

mkey ::= '"' mfield '"'
skey ::= '"' sfield '"'
inputstring ::= [^*]* // Zero or more of any character, except asterisk.
applykey ::= [^_]+ // One or more of any character, except underscore.

Course Offerings Fields

When querying course_offerings kind, use these fields:

mfield ::= 'avg' | 'pass' | 'fail' | 'audit' | 'year'
sfield ::= 'title' | 'dept' | 'code' | 'instructor'

Facilities Fields

When querying facilities kind, use these fields:

mfield ::= 'lat' | 'lon' | 'seats'
sfield ::= 'address' | 'building' | 'furniture' | 'href' | 'name' | 'number' | 'type'

Aggregation

The query language supports performing calculations across a group of results.

Supported Aggregation Functions:

• MAX: Find the maximum value of an mfield. Returns the same number that is in the originating dataset.
• MIN: Find the minimum value of an mfield. Returns the same number that is in the originating dataset.
• AVG: Find the average value of an mfield. Returns a number rounded to two decimal places.
• SUM: Find the sum of an mfield. Returns a number rounded to two decimal places.
• COUNT: Count the number of unique occurrences of a field. Returns whole numbers.

Requirements:

• MAX/MIN/AVG/SUM should only be requested for numeric keys (mfield). COUNT can be requested for all keys.
• The applykey in an APPLYRULE should be unique, so no two APPLYRULEs should share an applykey with the same name.
• If GROUP is present, all COLUMNS keys must correspond to one of the GROUP keys or to applykeys defined in the APPLY block.

Sort

The query language supports sorting by:

• A single column as in V1, e.g., "ORDER": "avg"
• Multiple columns with direction by creating an object to sort by ascending or descending order and by multiple columns, e.g., "ORDER": {"dir": "DOWN", "keys": ["maxSeats"]}

Direction (dir): The order of the sorting is set by the direction:

• "UP": Sort results ascending
• "DOWN": Sort results descending

Keys: The keys field allows for sorting by multiple keys (i.e., columns), where each additional key resolves ties for the previous key.

Examples:

• "keys": ["avg"]: sorts by a single key
• "keys": ["year", "avg"]: sorts by multiple keys. In this case, the average should be used to resolve ties for entries in the same year

Requirements:

• All SORT keys must also be in the COLUMNS.
• When sorting produces tied values (and no additional keys are specified to break ties), the relative order of those entries is not guaranteed.

Wildcards

You can use optional asterisks (*) in IS comparisons to match patterns. For example, "IS": {"dept": "C*"} matches any department that starts with "C". Because the asterisks can appear at the beginning and/or end, there are four possible forms:

inputstring: Matches inputstring exactly
*inputstring: Ends with inputstring
inputstring*: Starts with inputstring
*inputstring*: Contains inputstring

Asterisks in the middle are not allowed (for example, input*string).

Results are ordered by id ascending to ensure stable pagination.

If the course already exists, the entire course is replaced; otherwise it is created.

The course and all of its sections are permanently removed. The response includes the course metadata and a count of sections that were removed.

Results are ordered by id ascending to ensure stable pagination.

If the section already exists, the entire section is replaced; otherwise it is created.

The section is permanently removed and its data is returned.

Results are ordered by id ascending to ensure stable pagination.

If the building already exists, the entire building is replaced; otherwise it is created.

The building and all of its rooms are permanently removed. The response includes the building metadata and a count of rooms that were removed.

Results are ordered by id ascending to ensure stable pagination.

If the room already exists, the entire room is replaced; otherwise it is created.

The room is permanently removed and its data is returned.