aidbox.bulk data import

Overview

aidbox.bulk API uses Aidbox JSON RPC API

Features of the aidbox.bulk API:

Asynchronous validation of uploaded resources & references
Human readable validation errors
Can process Aidbox & FHIR format

RPC methods

`aidbox.bulk/import-start`

Starts aidbox.bulk import.

Object with the following structure:

format input resources format, fhir or aidbox (default: fhir)
meta meta data that will be attached to each of the imported resources
onconflict action to resolve id uniqueness constraint violation, update or override
input array of objects
url string with input source url

Returns input params object with following attributes added:

status inprogress, finished, failed
count overall number of imported resources
time import execution time
input the import status of each specific input
status loaded, validated, failed
count number of loaded resources
errors number of validation errors
time input execution time

message "There is running importwait until it finish or cancel it with aidbox.bulk/importcancel"
import object identical to result object

Resource requirements for `aidbox.bulk/import-start`:

Operation id resourceType
aidbox.bulk/import-start Required Required

Operation	id	resourceType
`aidbox.bulk/import-start`	Required	Required

Example

yaml

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-start
params:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz'}

status: 200

yaml

result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta: {source: app1}
  input:
  - {url: 'https://storage.googleapis.com/aidbox-public/synthea/100/corrupted-patient.ndjson.gz', status: loaded, count: 124, errors: 1, time: 157}
  status: failed
  errors: 1
  time: 241

`aidbox.bulk/import-status`

Returns latest aidbox.bulk import info

_Expects no parameters_

Same as aidbox.bulk/import-start result

message "No active imports"

Example

yaml

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-status
params: {}

Status: 200

yaml

result:
  on-conflict: update
  id_prefix: app1
  format: fhir
  meta:
    source: app1
  input:
    - url: >-
        https://storage.googleapis.com/aidbox-public/synthea/100/Patient.ndjson.gz
      status: loaded
      count: 124
      errors: 0
      time: 1205
  status: finished
  count: 0
  time: 1452

`aidbox.bulk/import-errors`

Returns latest aidbox.bulk import detailed errors list

Object with the following structure:

omitresources? (default: false)

Result is an array of objects with following structure:

input_no
line_no
type
id
resource
errors

message "No active imports"

Example

yaml

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-errors
params:
  omit-resources?: true

Status: 200

yaml

result:
  - input_no: 0
    line_no: 1
    type: Patient
    id: e9adac47-eb98-4fce-b871-512226086c97
    errors:
      - path:
          - name
          - 0
          - given
        message: expected array
  - input_no: 0
    line_no: 20
    type: Patient
    id: b2d58f0f-4499-4392-ad3a-1c2141c8a6c1
    errors:
      - path:
          - address
          - 0
          - line
        message: expected array

`aidbox.bulk/import-cancel`

Cancels latest aidbox.bulk import

_Expects no parameters_

Returns object with following structure:

message "Import canceled"
import Same as aidbox.bulk/importstart result

message "No active imports"

Example

yaml

POST /rpc
content-type: text/yaml
accept: text/yaml

method: aidbox.bulk/import-cancel

Status: 200

result:
  message: "Import Canceled"
  import: <. . .>

Import local file

Sometimes you want to import local file into local Aidbox. Possible solutions for local development:

Add volume to the `aidboxone` container (not `aidboxdb`):

volumes:
- ./Encounter.ndjson.gz:/resources/Encounter.ndjson.gz
# url: file:///resources/Encounter.ndjson.gz

Use tunneling e.g. ngrok:&x20;

python3 -m http.server  ngrok http 8000 # url: https://<...>.ngrok-free.app/Encounter.ndjson.gz

aidbox.bulk data import

Overview

RPC methods

aidbox.bulk/import-start

Resource requirements for aidbox.bulk/import-start:

Example

aidbox.bulk/import-status

Example

aidbox.bulk/import-errors

Example

aidbox.bulk/import-cancel

Example

Import local file

Add volume to the aidboxone container (not aidboxdb):

Use tunneling e.g. ngrok:&x20;

`aidbox.bulk/import-start`

Resource requirements for `aidbox.bulk/import-start`:

`aidbox.bulk/import-status`

`aidbox.bulk/import-errors`

`aidbox.bulk/import-cancel`

Add volume to the `aidboxone` container (not `aidboxdb`):