Skip to content

Commit

Permalink
Browse files Browse the repository at this point in the history
tutorial update
  • Loading branch information
xperthunter committed Jun 7, 2022
1 parent b8d678d commit 1579583
Showing 1 changed file with 61 additions and 54 deletions.
115 changes: 61 additions & 54 deletions TUTORIAL.md
View file
Expand Up @@ -14,14 +14,16 @@ The first entry point to look for guidance on SpecDB functions is to use the hel
6. `specdb backup --db --backup`
7. `specdb restore --backup --backup`

The subcommands listed above in the logical order the commands are used in. Each subcommand has a separate help menu from `specdb --help` that can be accessed, (e.g `specdb forms --help`). Users first need to create a SpecDB SQLite database file with `create`. Next, users need to populate the database with information, the `forms` command makes the forms for the data fields needed for the SpecDB schema. With a filled form, users use `insert` to insert the form into their database. To verify/check what they inserted, users can use `summary` to investigate the contents of any SpecDB table. Users can pull data out of the database with `query`. With `query` users provide a SQL SELECT statement on the SpecDB summary view to pull data out of the database. Commands `backup` and `restore` are for the incremental backup operations.
The subcommands listed above in the logical order the commands are used in. Each subcommand has a separate help menu from `specdb --help` that can be accessed, (e.g `specdb forms --help`). Users first need to create a SpecDB SQLite database file with `create`. Next, users need to populate the database with information. The `forms` command makes the forms for the data fields needed for the SpecDB schema. With a filled form, users use `insert` to insert the form into their database. To verify/check what they inserted, users can use `summary` to investigate the contents of any SpecDB table. Users can pull data out of the database with `query`. With `query` users provide a SQL SELECT statement on the SpecDB summary view to pull data out of the database. `query` also allows for users to extract records from SpecDB by row-ids in the `summary` table. Commands `backup` and `restore` are for the incremental backup operations.

## 2. Instantiating a new SpecDB database

A SpecDB database can be created with `specdb create`. The location of the database .db file needs to be known by the users using the database, or an environment variable can be set. In this tutorial we are going to create a database for the sample data stored in this repository `./sample/`.

```
$ specdb create --db sample.db --backup sample.backup.db
db configured and initalized
Done
```

## 3. Making forms
Expand All @@ -31,19 +33,21 @@ Now that a database has been made, we can start adding data to the database. To
Running `specdb forms --table user` results in the user form being printed to screen. The output should be exactly as seen below.

```
user: # provide information about a user, REQUIRED: `user_id`
user_id: '' # no spaces, must be unique, len <= 8, Ex: KJF
given_name: '' # len <= 64, Ex: Keith
family_name: '' # len <= 64, Ex: Fraga
middle_initials: '' # len <= 16, Ex: J
department_and_institution: '' # len <= 128, Ex: MCB UCD
country: '' # len <= 32, Ex: USA
state_province: '' # len <= 32, Ex: NY
city: '' # len <= 32, Ex: Troy
postal_code: '' # len <= 32, Ex: 12180
role: '' # len <= 64, Ex: postdoc
organization_type: '' # len <= 64, Ex: academic
email_address: '' # no spaces, must have `@`, len <= 64, Ex: user@mail.com
user: # provide information about a user, REQUIRED: `user_id`
0:
user_id: # no spaces, must be unique, len <= 8, Ex: KJF
given_name: # len <= 64, Ex: Keith
family_name: # len <= 64, Ex: Fraga
middle_initials: # len <= 16, Ex: J
department_and_institution: # len <= 128, Ex: MCB UCD
country: # len <= 32, Ex: USA
state_province: # len <= 32, Ex: NY
city: # len <= 32, Ex: Troy
postal_code: # len <= 32, Ex: 12180
role: # len <= 64, Ex: postdoc
organization_type: # len <= 64, Ex: academic
email_address: # no spaces, must have `@`, len <= 64, Ex: user@mail.com
```

To fill the form, the output of the `specdb forms` command can be redirected to any file. (Note: the file extension is not strict, tested with `txt`, `yaml`, `yml`. )
Expand Down Expand Up @@ -102,30 +106,30 @@ user: # # provide information about a user, REQUIRED: `user_id`

This output can be redirected to any file and filled in the with appropriate information.

Forms can be made for different tables to be in the same form. A very common use for this is inserting a buffer into SpecDB. There are two tables in the SpecDB schema that describe buffers, `buffer` and `buffer_components`. `buffer` provides the name of the buffer, the `buffer_id` and the buffer's pH. The `buffer_components` table allows for variable number of components to be added to the buffer. For example, if a buffer with 3 components needed to be added to SpecDB, the following command would generate the correct form:
Forms can be made for different tables to be in the same form. A very common use for this is inserting a buffer into SpecDB. There are two tables in the SpecDB schema that describe buffers, `buffer` and `buffer_components`. `buffer` provides the name of the buffer, the `buffer_id` and the buffer's pH. The `buffer_components` table allows for variable number of components to be added to the buffer. For example, if a buffer with 3 components is needed to be added to SpecDB, the following command would generate the correct form:

```$ specdb forms --table buffer buffer_components --num 1 3
buffer components go in the `buffer_components` table
buffer: # store all buffers used, their id and their pH, REQUIRED: `buffer_id`, `buffer_pH`, NOTE: the buffer components go in the `buffer_components` table
0:
buffer_id: '' # text identifier for the buffer, must be unique, no spaces, len <= 32, Ex: hn4071
buffer_ph: 0.0 # pH of buffer, must be a number, Ex: 8.1
buffer_comment: '' # free text field for any notes about the buffer, len <= 128, Ex: made by KJF 12/04
buffer_components: # describe the component(s) of a buffer, REQUIRED: `buffer_id`, `buffer_component`, `buffer_component_value`, `buffer_component_unit`
buffer_id: # text identifier for the buffer, must be unique, no spaces, len <= 32, Ex: hn4071
buffer_ph: # pH of buffer, must be a number, Ex: 8.1
buffer_comment: # free text field for any notes about the buffer, len <= 128, Ex: made by KJF 12/04
buffer_components: # describe the component(s) of a buffer, REQUIRED: `buffer_id`, `buffer_component`, `buffer_component_value`, `buffer_component_unit`
0:
buffer_id: '' # `buffer_id` this component goes to, must already be in buffer table, Ex: hn4071
buffer_component: '' # name of the component, len <= 64, Ex: NaCl
buffer_component_value: 0.0 # the numeric concentration value of the component, Ex: 100.0
buffer_component_unit: '' # unit of concentration, one of (`mM`, `% (v/v)`, `mg/mL`), Ex: mM
buffer_id: # `buffer_id` this component goes to, must already be in buffer table, Ex: hn4071
buffer_component: # name of the component, len <= 64, Ex: NaCl
buffer_component_value: # the numeric concentration value of the component, Ex: 100.0
buffer_component_unit: # unit of concentration, one of (`mM`, `% (v/v)`, `mg/mL`), Ex: mM
1:
buffer_id: '' # `buffer_id` this component goes to, must already be in buffer table, Ex: hn4071
buffer_component: '' # name of the component, len <= 64, Ex: NaCl
buffer_component_value: 0.0 # the numeric concentration value of the component, Ex: 100.0
buffer_component_unit: '' # unit of concentration, one of (`mM`, `% (v/v)`, `mg/mL`), Ex: mM
buffer_id: # `buffer_id` this component goes to, must already be in buffer table, Ex: hn4071
buffer_component: # name of the component, len <= 64, Ex: NaCl
buffer_component_value: # the numeric concentration value of the component, Ex: 100.0
buffer_component_unit: # unit of concentration, one of (`mM`, `% (v/v)`, `mg/mL`), Ex: mM
2:
buffer_id: '' # `buffer_id` this component goes to, must already be in buffer table, Ex: hn4071
buffer_component: '' # name of the component, len <= 64, Ex: NaCl
buffer_component_value: 0.0 # the numeric concentration value of the component, Ex: 100.0
buffer_component_unit: '' # unit of concentration, one of (`mM`, `% (v/v)`, `mg/mL`), Ex: mM
buffer_id: # `buffer_id` this component goes to, must already be in buffer table, Ex: hn4071
buffer_component: # name of the component, len <= 64, Ex: NaCl
buffer_component_value: # the numeric concentration value of the component, Ex: 100.0
buffer_component_unit: # unit of concentration, one of (`mM`, `% (v/v)`, `mg/mL`), Ex: mM
```

Expand Down Expand Up @@ -196,31 +200,33 @@ ensure all ids this table relates to (i.e constructs relate to targets) are inse
Aborting
```

This insertion results in a different error from before. Here, the error was not because the information was new because the `--write` was on. The error is coming from SQLite that foreign key constraint failed. A foreign key is a relationship between two tables. These foreign keys need to be present in the two tables for SpecDB to allow an insert. The error message is not clear on what column the constrain failure came on, but it says the error came from the `target` table and to inspect the form for what is required for a target. Using `forms` we can see what is required for `target`.
This insertion results in a different error from before. Here, the error was not because the information was new because the `--write` was on. The error is coming from SQLite, that a foreign key constraint failed. A foreign key is a relationship between two tables. These foreign keys need to be present in the two tables for SpecDB to allow an insert. The error message is not clear on what column the constrain failure came on, but it says the error came from the `target` table and to inspect the form for what is required for a target. Using `forms` we can see what is required for `target`.

```
$ specdb forms --table target
target: # molecular target information, REQUIRED: `target_id`
target_id: '' # text identifier for target, must be unique, no spaces, len <= 32, Ex: Db0515
target_comment: '' # free field comment, anything to note about target, len <= 128, Ex: from UW-Madison
target_sequence: '' # target's molecular seq., any type of code/alphabet, no spaces, len <= 1024, Ex: MGSHHHHILVAM
organism_source: '' # organism name for target source, can indicate if target is synthetic, len <= 128, Ex: synthetic
gene_name: '' # gene name, len <= 64, Ex: SpikeCoV2
project_id: '' # `project_id` target is a member of, must be in project table, Ex: SpikeFraga
target_preparer: '' # `user_id` that assigned the target, must be in user table, Ex: KJF
target: # molecular target information, REQUIRED: `target_id`
0:
target_id: # text identifier for target, must be unique, no spaces, len <= 32, Ex: Db0515
target_comment: # free field comment, anything to note about target, len <= 128, Ex: from UW-Madison
target_sequence: # target's molecular seq., any type of code/alphabet, no spaces, len <= 1024, Ex: MGSHHHHILVAM
organism_source: # organism name for target source, can indicate if target is synthetic, len <= 128, Ex: synthetic
gene_name: # gene name, len <= 64, Ex: SpikeCoV2
project_id: # `project_id` target is a member of, must be in project table, Ex: SpikeFraga
target_preparer: # `user_id` that assigned the target, must be in user table, Ex: KJF
```

We there are relationships between the `user` table and the `project` table. Inspecting what we tried to add for the target in `sample.yaml` we see:
There are relationships between the `user` table and the `project` table. Inspecting what we tried to add for the target in `sample.yaml` we see:

```
target:
target_id: Db0515
target_comment: ''
target_sequence: MGPLIEVLA
organism_source: synthetic
gene_name: ''
project_id: DBh
target_preparer: GTM
target: # molecular target information, REQUIRED: `target_id`
0:
target_id: Db0515 # text identifier for target, must be unique, no spaces, len <= 32, Ex: Db0515
target_comment: # free field comment, anything to note about target, len <= 128, Ex: from UW-Madison
target_sequence: MGPLIEVLA # target's molecular seq., any type of code/alphabet, no spaces, len <= 1024, Ex: MGSHHHHILVAM
organism_source: synthetic # organism name for target source, can indicate if target is synthetic, len <= 128, Ex: synthetic
gene_name: # gene name, len <= 64, Ex: SpikeCoV2
project_id: DBh # `project_id` target is a member of, must be in project table, Ex: SpikeFraga
target_preparer: GTM # `user_id` that assigned the target, must be in user table, Ex: KJF
```

We inserted the user information for `GTM` but not the project id `DBh` this target is associated with. We can solve this by inserting the project information using the `sample/sample_forms/project.yaml` form.
Expand All @@ -247,10 +253,11 @@ Inserted data from json file at /Users/kfraga/RESEARCH/SpecDB/code/sample/sample
Inserting FIDs into SpecDB are typically organized into sessions. When FIDs are recorded from a Bruker instrument, they are recorded into sub-directories of a data collection session directory. We mimiced this situation in `sample/sample_sessions/test00/`. In `test00/`, there are two subdirectories, each with a time domain dataset. There is also a `specdb.session.yaml`. This file can be creatd with `specdb forms --table session`. The session yaml describe thes spectrometer this data collection was at, the user who conducted the session, and the sample tube the data was collected at. Contents of this session is displayed below:

```
session:
session_preparer: TAR
spectrometer_id: Hu800
pst_id: Db0515A.000
session: # describe a data collection session, REQUIRED: `session_preparer`, `spectrometer_id`, `pst_id`
0:
session_preparer: KJF # `user_id` of the person who prepared and collected the session, must already be in user table, Ex: KJF
spectrometer_id: Hu800 # `spectrometer_id` session was collected at, must already be in spectrometer table, Ex: Hu800
pst_id: Db0515A.000 # `pst_id` the session is for
```

All three items in this session are required.
Expand Down

0 comments on commit 1579583

Please sign in to comment.