The command line
gqsar-cli.exe runs workflows, templates, the Dataset Builder and the model commands without a window, for batch work, servers and reproducible scripts.
The installation folder holds two programs: GeinQSAR.exe, the desktop application, and gqsar-cli.exe, its command-line counterpart. Both use the same chemistry, descriptor, modelling and curation code, so a workflow saved on the canvas gives the same numbers when it is run from the command line, and a dataset built with gqsar-cli fetch is the same table the Dataset Builder writes. The command line is the right tool when the same analysis has to be repeated on many files, run overnight, run on a machine without a display, or recorded in a lab notebook as an exact command.
Open a Command Prompt or PowerShell window in the installation folder (or add the folder to the PATH) and type gqsar-cli --help for the full summary of commands and options.
Conventions
- Every call has the form
gqsar-cli <command> [options]. Options are written--name valueor--name=value;-iand-oare short for the input and output file. - Results that another program reads (written file paths, tables, JSON) go to standard output. Progress, notes and error messages go to standard error, so
gqsar-cli ... > list.txtcaptures the file list and still shows the progress on screen. - An option the command does not know is refused with exit code 1 and a message such as
unknown option --respnse for 'train'. A misspelt option is therefore never silently ignored. - Relative paths resolve against the current directory. The workflow templates refer to their sample data as
data/..., so run them from the installation folder or edit the path with--set. - Ctrl+C cancels a running command cleanly; it then ends with exit code 4.
The commands
| Command | What it does |
|---|---|
run | Executes a workflow file (.gqflow) headlessly. |
templates | Lists the ready-made workflow templates or writes them as .gqflow files. |
nodes | Lists the workflow node catalogue, optionally with ports and parameters. |
descriptors | Computes molecular descriptors for a structure file into a CSV table. |
train | Fits a model on a descriptor table and saves the model bundle. |
predict | Applies a saved model bundle to a structure file or a descriptor table. |
design | Designs a library around a scaffold or a hit, ranks it with your models and writes the ranked table (Designing a library). |
convert | Converts between structure file formats. |
project | Describes a .gqproj project and, with --verify, checks its integrity. |
fetch | Builds a dataset from ChEMBL or PubChem: search, download and curation. |
curate | Curates a local activity table, or replays a recorded curation. |
version | Prints the version, the workflow format version, the number of node types and descriptors, and the number of hardware threads. |
Running a workflow
Any workflow built on the workflow canvas or by a wizard can be saved as a .gqflow file and run again from the command line:
gqsar-cli run --flow F.gqflow [--set node.param=value]... [--target NODE]
[--out DIR] [--report FILE] [--seed N] [--threads N]
[--no-cache] [--quiet] [--describe]
| Option | Effect | When to use it |
|---|---|---|
--flow F.gqflow | The workflow to run. The file name may also be given without the option. | Always. |
--set node.param=value | Changes one parameter of one node before the run. The node is named by its id (the short name shown with --describe), the parameter by its key. The value is checked against the parameter type; an unknown node or parameter stops the run with exit code 1. May be repeated. | To run one workflow on another input file, with another test fraction or another model setting, without editing the file. |
--target NODE | Runs only that node and the nodes it depends on. May be repeated. | To check the first steps of a long workflow (reading, curation, split) before the expensive ones. |
--out DIR | Relative output paths of the writer, report and export nodes resolve against DIR (inputs still resolve against the current directory). The run summary, with every node's state, log and a preview of its data, is written to DIR/run.json. | For every batch run, so that each run keeps its own folder. |
--report FILE | Writes the report of the first model the run produced as a PDF, with the whole workflow embedded (OECD principle 2: an unambiguous algorithm). | When the result is to be archived or sent. |
--seed N | The random seed of the run (a non-negative integer below 264). | To repeat a run exactly, or to measure how much a result depends on the random split. |
--threads N | At most N nodes execute at the same time; 1 runs them strictly one after another. | On a shared machine, or to keep memory use down on large data. |
--no-cache | Recomputes every node instead of reusing results of an identical earlier computation. | When timing a run, or after changing an input file in place. |
--quiet | Suppresses the progress lines on standard error. | In scripts that log only failures. |
--describe | Prints the nodes, their parameters and connections after any --set, and exits without running. | To find node ids and parameter keys, and to check what a --set changed. |
While the run proceeds, each node reports a line on standard error: [run ] when it starts, [ ok ] with its time when it finishes (and (cached) when the result was reused), [FAIL] with the reason, or [skip] when a node could not run because an earlier one failed. Warnings of a node are printed under its line. A summary table closes the run, and the paths of every file written are printed on standard output.
Templates and the node catalogue
gqsar-cli templates lists the ready-made workflows of the template gallery; --verbose adds the description and the sample dataset each one uses. gqsar-cli templates --write DIR writes every template as DIR/<id>.gqflow, ready to be run or edited.
| Template id | Title | Sample data |
|---|---|---|
qsar2d_oecd | 2D-QSAR (PLS) OECD workflow | data/flow/ache_pic50.csv (429 compounds) |
auto_model_search | Automatic model search | data/flow/ache_pic50.csv (429 compounds) |
qsar_classification | 2D-QSAR classification (blocker / non-blocker) | data/flow/herg_class.csv (554 compounds) |
qsar3d_kefra | KEFRA 3D-QSAR | data/qsar3d/steroids.csv (31 compounds) |
qsar3d_atom | Atom-based QSAR | data/qsar3d/steroids.csv (31 compounds) |
chirality_cliffs | Chirality cliff analysis | data/desc/chirality_cliffs.csv (34 compounds) |
virtual_screening | Virtual screening with model + AD | data/flow/ache_pic50.csv (429 compounds) |
library_design | LIBQORA library design | data/flow/ache_pic50.csv (429 compounds) |
gqsar-cli nodes lists the node types by category (Input, Structure preparation, Descriptors, Data handling, Modelling, Validation, Prediction, 3D-QSAR, Analysis, Report, Output) with their ids and labels. --verbose adds each node's input ports, output ports and parameters with their types, which is what a --set needs; --json prints the whole catalogue as JSON.
Descriptors, models and conversion
descriptors
gqsar-cli descriptors -i in.sdf -o out.csv [--preset ID | --families a,b | --ids x,y]
Reads any structure file the program reads (see File formats) and writes one row per molecule and one column per descriptor. --families takes family ids such as crippen,tpsa,hbond; --ids takes single descriptor ids; --preset takes chirality_aware_2d or chirality_aware_3d. Without a selection the preset chirality_aware_2d is used and a note says so. Without -o (or with -o -) the table goes to standard output. Records that cannot be read are reported as notes (the first five) and skipped.
train and predict
gqsar-cli train -i table.csv --response pIC50 --model pls [--params '{"components":3}']
[--id-column id] [--smiles-column smiles] -o model.json
gqsar-cli predict --model model.json -i compounds.sdf -o predictions.csv
train fits one model on a descriptor table. The response column holds numbers for a regression model and class labels for a classifier; every other numeric column is used as a descriptor, and non-numeric cells count as missing. --model takes a model id (default pls); the regression ids include mlr, stepwise_mlr, pls, pcr, ridge, elastic_net, svr, gpr, knn_regressor, random_forest_regressor, extra_trees_regressor, gbm_regressor and mlp_regressor; the classifier ids include logistic_regression, lda, qda, svc, knn_classifier, random_forest_classifier, extra_trees_classifier, gbm_classifier, gaussian_nb, bernoulli_nb and mlp_classifier. --params passes the model's parameters as a JSON object; a parameter outside its range is refused with its permitted interval.
predict accepts either a structure file, for which the model's descriptors are computed by id, or a descriptor table (.csv or .tsv) whose header contains every descriptor of the model. Descriptor transformations stored in the bundle (imputation, outlier limits, scaling) are applied automatically, so the input must hold descriptors as computed, not scaled ones. The output columns are:
| Model | Columns written |
|---|---|
| Regression | id, observed, predicted, residual, standard_error, in_domain, ad_reasons |
| Classifier | id, observed, predicted (class labels), one probability_<label> column per class (or one decision_<a>_vs_<b> column per pair of classes for a model without probabilities), in_domain, ad_reasons. The observed class is filled in when the input holds the model's response column. |
predict then prints a warning that the predictions and domain verdicts are not reliable unless the input was transformed in exactly the same way. Train such a model again.convert
gqsar-cli convert -i in.smi -o out.sdf [--format sdf] [--coords 2d|none]
The output format follows the extension of -o unless --format names it. Writing SDF, MOL, MOL2, PDB, PDBQT or XYZ from an input without coordinates (SMILES, CSV) lays the molecules out in 2D first, so that the file can be depicted instead of carrying an atom block of zeroes. --coords none writes the connectivity with zero coordinates; --coords 2d lays out again even molecules that already have 2D coordinates. Existing 3D coordinates are never replaced.
design
gqsar-cli design --scaffold "NC(=O)c1ccc([*:1])cc1" --classes alkyl,halogen --model m.json ^
--filter lipinski,pains --objectives "pIC50 maximise; QED maximise weight=0.5" -o library.csv
gqsar-cli design --decorate hit.sdf --sites "[cH]" --model m.json --calibration cal.csv ^
--activity pIC50 --alpha 0.2 --objectives "pIC50 maximise" --select 24 -o library.csv --sdf library.sdf
Exactly one of --scaffold and --decorate is required, and so is --objectives: a design with no objective has no order. The substituents come from --classes of the built-in library or from --substituents FILE; --filter names the rules to apply, and none is applied unless it is named; --model may be given more than once, each model getting a column of its own; --calibration with --activity, --alpha and --interval calibrate the prediction interval; --rank desirability|pareto, --select N and --from-top N set the ranking and the diverse selection; --max, --sample, --seed, --prefix and --stereo control the enumeration. --alpha, --interval or --activity without --calibration is refused rather than ignored, because a setting that has no effect is a setting that misleads. -o - writes the table to standard output.
project
gqsar-cli project P.gqproj prints what a project contains. With --verify it checks every entry against the checksum recorded when the project was saved, prints integrity: N of M entries verified, lists each problem, and ends with exit code 3 if anything does not match.
The Dataset Builder on the command line
fetch and curate are the command-line form of the Dataset Builder. A run with -o name.csv writes name.csv (the curated table), name.sdf, name.raw.json (the raw records), name.curation.json (the provenance), name.removed.csv (each removed record with its reason) and the curation report as name.report.txt and name.report.pdf, with --report-word and --report-excel adding the other two forms; the files are described on File formats. The funnel (records in, out and removed per step), the findings and the range of the response are printed on standard error, and the paths of the files on standard output.
Searching and downloading
| Command | What it does |
|---|---|
fetch chembl-targets --query TEXT [--organism NAME] [--limit N] [--json] | Searches ChEMBL by name, gene symbol, UniProt accession or ChEMBL id and lists each target with the number of activities per type (IC50, Ki and so on). Use it to find the target id and to see which activity type has enough data. |
fetch chembl --target CHEMBLxxx --type IC50 --preset regression -o out.csv | Downloads the activities of one target and curates them. Without --type all types are downloaded and the most frequent potency type is chosen; several types need --allow-mixed-types. |
fetch chembl-document --id CHEMBLxxxx [--type Ki] [--preset series3d] -o out.csv | Downloads the activities reported in one publication, which is usually a congeneric series and the natural input for 3D-QSAR. The default preset here is series3d. |
fetch chembl-molecule --id CHEMBLxxx | Prints the name, SMILES, InChIKey, formula and molecular weight of one ChEMBL compound. |
fetch chembl-status | Prints the ChEMBL release, its date and the numbers of activities, compounds and targets. |
fetch pubchem-assays --gene SYMBOL | --protein ACCESSION | Lists the PubChem assays of a gene or protein (25 by default, --limit N). --organism, --potency-only (assays with an active concentration), --largest-first and --min-compounds N narrow the list. |
fetch pubchem-assay --aid N --info | Describes one assay: source, method, targets, counts of active and inactive compounds, the activity readout and its unit, and the targets and activity types found in the rows of its data table. |
fetch pubchem-assay --aid N [--target KEY] [--preset ...] -o out.csv | Downloads and curates one assay. A table with rows for several targets needs --target; rows of different targets are never pooled. For an active/inactive screen use --preset classification --labels-from-outcome. |
fetch pubchem-compounds --names FILE | --cids FILE | --smiles FILE | --inchikeys FILE -o out.csv | Looks up compounds by name, CID, SMILES or InChIKey (one per line; a comma-separated list is also accepted for CIDs and InChIKeys). Without --preset it writes the lookup table; with --preset library it curates the structures as a screening library. |
Curating a local file and replaying a curation
gqsar-cli curate -i raw.csv --smiles-col C --value-col C --unit nM --preset P -o out.csv
gqsar-cli curate --replay out.curation.json [--refetch]
curate -i reads a CSV, TSV or SD file (or a raw-record snapshot .raw.json). Columns are recognised automatically; --smiles-col, --id-col, --name-col, --value-col, --relation-col, --unit-col, --type-col, --assay-col, --ref-col and --year-col override the guess, --delimiter fixes the separator (tab for a tab) and --decimal-comma yes reads 1,5 as 1.5. A file without a value column is curated as a structure library.
curate --replay runs a recorded curation again from its provenance file and compares the SHA-256 checksum of the new table with the recorded one. An identical result ends with exit code 0 and the message replay is identical to the recorded run; a different one lists the differences and ends with exit code 3. With --refetch the source is queried again, so that the run shows what a newer database release gives (see Provenance and replay).
Curation options
--preset chooses the study type, which sets every curation step at once: regression, classification, series3d or library. The options below change single steps of the preset; --options FILE.json reads a whole option set, and a .curation.json provenance file is accepted there too, so a curation can be applied unchanged to new data.
| Group | Options |
|---|---|
| Endpoint | --type T[,T] (activity types), --allow-mixed-types |
| Classes | --threshold P (class boundary on the p-scale), --grey-zone W (records within the zone are left out), --labels-from-outcome (use the active/inactive outcome of the source), --keep-censored or --exact-only |
| Units | --unit U (assumed when a record has none), --convert-mass-units, --require-pchembl, --no-pchembl-check, --remove-pchembl-mismatch |
| Validity | --keep-flagged, --keep-outside-typical-range, --drop-outside-typical-range, --keep-potential-duplicates |
| Target | --select-target ID|ACCESSION|NAME for records that cover several targets |
| Assay | --confidence N (ChEMBL confidence score 0 to 9), --assay-types B,F, --organism NAME, --keep-variants, --document ID, --assay ID, --one-document, --one-assay, --dominant-scaffold, --generic-scaffold |
| Structure | --stereo keep|flatten, --remove-undefined-stereo, --tautomer, --mixtures remove|keep-largest, --keep-isotopes, --no-neutralize, --max-mw N, --min-mw N, --min-heavy N, --max-heavy N, --elements C,N,O |
| Replicates | --aggregate median|mean|geomean, --max-spread X, --keep-repeats |
| Response | --min-p X, --max-p Y |
| Output | --no-sdf, --no-report, --raw FILE (the raw records as CSV), --json (the provenance as JSON on standard output), --quiet, --title TEXT |
The confidence threshold of the presets (8) suits single proteins. When the records of a ChEMBL download belong to a protein complex or family, the threshold is lowered to suit the target type and a note says so; --confidence fixes it.
Network options
| Option | Effect |
|---|---|
--cache DIR | Keeps every answer of ChEMBL and PubChem in DIR. A repeated command is then fast and does not depend on the network. |
--offline | Reads only from the cache (needs --cache). A request that was never recorded fails with a message saying so. |
--refresh | Asks the services again and replaces the cached answers. |
--timeout SECONDS | Limit for one request (the default total is 300 seconds). |
--retries N | Number of repeated attempts after a failed request (default 4). |
--proxy HOST:PORT | Uses this proxy instead of the one configured in Windows; direct uses none. A proxy that asks for a user name and password is not supported. |
--max-records N | Stops the download after N records, for a quick look at a large target. |
--chembl-url URL, --pubchem-url URL | Another address of the same service, such as a local mirror. |
Exit codes
| Code | Meaning | Typical cause |
|---|---|---|
| 0 | Success | The command did what was asked. |
| 1 | Usage | A missing or unknown option, a value of the wrong type, an unknown node or parameter in --set. |
| 2 | Input | An unreadable or missing file, an unknown identifier (a ChEMBL id that does not exist), a provenance file that cannot be read. |
| 3 | Execution failure | A failed node, a network failure, nothing produced (no compound passed the curation), a write error, a replay that does not reproduce the recorded table, a project that fails verification. |
| 4 | Cancelled | Ctrl+C or a stop requested by the system. |
A batch script should test the exit code rather than parse the messages, which are written for people.
Examples
Write the templates, check the first steps of the PLS workflow with a 20% test set, then run it completely into its own folder with a report:
gqsar-cli templates --write flows
gqsar-cli run --flow flows\qsar2d_oecd.gqflow --set split.test_fraction=0.2 --target split
gqsar-cli run --flow flows\qsar2d_oecd.gqflow --set split.test_fraction=0.2 ^
--out results\ache --report results\ache\report.pdf --seed 42
Find a target, build a regression dataset with a response cache, and later rebuild it offline:
gqsar-cli fetch chembl-targets --query "acetylcholinesterase" --organism "Homo sapiens"
gqsar-cli fetch chembl --target CHEMBL220 --type IC50 --preset regression ^
--cache cache -o ache_ic50.csv
gqsar-cli curate --replay ache_ic50.curation.json --cache cache --offline
Compute descriptors, train a random forest and predict a library:
gqsar-cli descriptors -i training.sdf -o training_desc.csv --families crippen,tpsa,hbond,ecfp4_1024
gqsar-cli train -i training_desc.csv --response pIC50 --model random_forest_regressor -o rf.json
gqsar-cli predict --model rf.json -i library.sdf -o library_predictions.csv
train. When the descriptors are computed with gqsar-cli descriptors from an SD file, the activity is not copied into the descriptor table; join the two tables on the id column first, or build the model in a workflow, where the dataset node does the join.Start options of the desktop program
GeinQSAR.exe also accepts a few options, which are useful in a shortcut or when something has gone wrong:
| Option | Effect |
|---|---|
files | Projects (.gqproj), workflows (.gqflow) or datasets (.sdf, .mol, .csv, .tsv, .smi) opened at start-up. |
--reset-layout | Starts with the factory arrangement of the panels and the window placement. |
--new-instance | Starts a new process instead of passing the files to a program that is already running. |
--no-splash | Does not show the start-up screen. |
--theme light|dark|system | Overrides the theme for this session only. |
--log FILE | Appends the session log to FILE as well. |
--version, --help | Prints the version, or the list of options, and exits. |