Auspice will not collect datasets and narratives from the run location

Hi, I recently had to reinstall my nextstrain environement from scratch. However now, when I run auspice view I get the following error as if auspice cannot collect the data and naratives from the project location

Auspice server now running at http://localhost:4000
Serving auspice version 2.23.0
Looking for datasets in /home/kimberleyb/miniconda3/envs/nextstrain/lib/auspice/node_modules/auspice/data
Looking for narratives in /home/kimberleyb/miniconda3/envs/nextstrain/lib/auspice/node_modules/auspice/narratives

GET AVAILABLE returning locally available datasets & narratives
[warning] Couldn’t collect available dataset files (path searched: /home/kimberleyb/miniconda3/envs/nextstrain/lib/auspice/node_modules/auspice/data)
[warning] Couldn’t collect available narratives (path searched: /home/kimberleyb/miniconda3/envs/nextstrain/lib/auspice/node_modules/auspice/narratives)

please help. I also tried to reinstall auspice/update everything

Kim

Hi @KimB – what is the command you are running auspice with? Looking at the error message it seems you are running it without providing arguments indicating where it should look for dataset JSONs and narrarative markdown files:

auspice view --datasetDir <PATH> --narrativeDir <PATH>

P.S. the latest auspice version is 2.29.0 (2.23.0 was from back in January) so you may want to upgrade!

Hi.

I have only just run " auspice view". As I always have

It works fine in my Linux terminal and finds the path himself, but running via Ubuntu the command gives me this error message. So I should add the path where my dataset is? But I run the command from the same path where the the data folder, config folder and snakefile are located; the same path I run my snakefile command from with " snakemake -s snakefile.txt auspice/CV-A2.json".
Should the command “auspice view” then not immediatly look in there?

I update the environment using

conda activate nextstrain
conda update --all

However I feel nothing us updated as augur also gives me the same error when trying to color the field- years- I believe this was resolved in the new augur version.

I also remover the nextstrain env completely from my Linux and Ubuntu terminal and reinstalled everything from scratch. Should that have not also enabled installement of all the latest tools, like augur and auspice?

So apart from my question on auspice, how do I update augur and auspice via conda. Because the above command " conda update --all" does not seem to update anything. Nor when I reinstalled everything. In fact it caused the error mentioned for auspice.

Thankyou

Kim

It works fine in my Linux terminal and finds the path himself…

auspice view, without being given an explicit --datasetDir argument, is designed to use the ./auspice directory (if it exists) or fall back to the current working directory. However this may be platform / install specific. Your original message indicated that auspice was “Looking for datasets in /home/kimberleyb/miniconda3/envs/nextstrain/lib/auspice/node_modules/auspice/data”, which is not what was you want, so in this case the safest / easiest solution is to explicitly provide a --datasetDir <PATH> argument to auspice view.

how do I update augur and auspice via conda.

For auspice:

conda activate nextstrain
npm update --global auspice # Auspice isn't on conda (yet)
auspice --version # current version is 2.29.1

For augur, since conda update --all didn’t seem to work, you could try

conda activate nextstrain
conda update -c conda-forge -c bioconda augur
# OR
conda install -c conda-forge -c bioconda augur=12
augur --version # latest version is 12.0.0

Hope this helps!
james

Ok thanks. I will give it a try.

Both update commands I also tried:

conda activate nextstrain
npm update --global auspice

conda activate nextstrain
conda update -c conda-forge -c bioconda augur

Neither seems to have updated the versions

But what you are saying is to add the version command as so ( checking as I am not a true bioinformaticion :slight_smile: )

conda activate nextstrain
npm update --global auspice
auspice --version 2.29.1

conda activate nextstrain
conda update -c conda-forge -c bioconda augur
augur --version 12.0.0

OR

conda activate nextstrain
conda install -c conda-forge -c bioconda augur=12
augur --version 12.0.0

Running the auspice --version command, or the augur --version command, will print out the current version of what you have installed, it won’t install or update anything.

From your original post, we can see that (a) you are using a conda environment named “nextstrain” and (b) auspice was installed using npm. So if auspice --version and augur--version are still indicating you are out-of-date (which means the instructions from my previous message didn’t work for some reason), then we can try to install the latest versions via:

conda activate nextstrain
npm install --global auspice@2.29.1
conda install -c conda-forge -c bioconda augur=12

Then run

auspice --version
augur --version

and let us know what those commands report?

updating auspice with npm install --global auspice@2.29.1

This is the report
npm WARN deprecated linspace@1.0.0: I wrote this module a very long time ago; you should use something else.
npm WARN deprecated redux-devtools@3.7.0: Package moved to @redux-devtools/core.
npm WARN deprecated request@2.88.2: request has been deprecated, see Request’s Past, Present and Future · Issue #3142 · request/request · GitHub
npm WARN deprecated core-js@2.6.12: core-js@<3.3 is no longer maintained and not recommended for usage due to the number of issues. Because of the V8 engine whims, feature detection in old core-js versions could cause a slowdown up to 100x even if nothing is polyfilled. Please, upgrade your dependencies to the actual version of core-js.
npm WARN deprecated redux-devtools-instrument@1.10.0: Package moved to @redux-devtools/instrument.
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated uuid@3.4.0: Please upgrade to version 7 or higher. Older versions may use Math.random() in certain circumstances, which is known to be problematic. See There’s Math.random(), and then there’s Math.random() · V8 for details.
npm WARN deprecated urix@0.1.0: Please see GitHub - lydell/urix: [DEPRECATED] Makes Windows-style paths more unix and URI friendly.
npm WARN deprecated resolve-url@0.2.1: GitHub - lydell/resolve-url: [DEPRECATED] Like Node.js’ `path.resolve`/`url.resolve` for the browser.
npm WARN deprecated querystring@0.2.0: The querystring API is considered Legacy. new code should use the URLSearchParams API instead.
npm WARN deprecated chokidar@2.1.8: Chokidar 2 will break on node v14+. Upgrade to chokidar 3 with 15x less dependencies.
npm WARN deprecated fsevents@1.2.13: fsevents 1 will break on node v14+ and could be using insecure binaries. Upgrade to fsevents 2.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@~2.3.2 (node_modules/auspice/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@2.3.2: wanted {“os”:“darwin”,“arch”:“any”} (current: {“os”:“linux”,“arch”:“x64”})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.2.7 (node_modules/auspice/node_modules/watchpack-chokidar2/node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for fsevents@1.2.13: wanted {“os”:“darwin”,“arch”:“any”} (current: {“os”:“linux”,“arch”:“x64”})
npm WARN react-addons-css-transition-group@15.6.2 requires a peer of react@^15.4.2 but none is installed. You must install peer dependencies yourself.

npm ERR! code EEXIST
npm ERR! syscall symlink
npm ERR! path …/lib/node_modules/auspice/auspice.js
npm ERR! dest /home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice
npm ERR! errno -17
npm ERR! EEXIST: file already exists, symlink ‘…/lib/node_modules/auspice/auspice.js’ → ‘/home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice’
npm ERR! File exists: /home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice
npm ERR! Remove the existing file and try again, or run npm
npm ERR! with --force to overwrite files recklessly.

npm ERR! A complete log of this run can be found in:
npm ERR! /home/kimberleyb/.npm/_logs/2021-07-26T11_53_10_321Z-debug.log

running auspice --version gives the following:

/home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice: line 2: syntax error near unexpected token newline' /home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice: line 2: Usage: npm ’

**augur update with **
conda install -c conda-forge -c bioconda augur=12

this is the report
Collecting package metadata (current_repodata.json): done
Solving environment: done

Package Plan

environment location: /home/kimberleyb/miniconda3/envs/nextstrain

added / updated specs:
- augur=12

The following packages will be SUPERSEDED by a higher-priority channel:

ca-certificates pkgs/main::ca-certificates-2021.7.5-h~ → conda-forge::ca-certificates-2021.5.30-ha878542_0
certifi pkgs/main::certifi-2021.5.30-py38h06a~ → conda-forge::certifi-2021.5.30-py38h578d9bd_0
openssl pkgs/main::openssl-1.1.1k-h27cfd23_0 → conda-forge::openssl-1.1.1k-h7f98852_0

Proceed ([y]/n)? y

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

running augur --version gives the following:

augur 12.0.0

so it seems that augur is ok now regarding the version, however I still get the same error with the config file colors.tsv where I indicated the field “year” with specified colors for each years. As indicated by emma, this was due to the augur version I was using. What was going wrong was the ‘validation’ step inside of ‘export’ which checks that the resulting files make sense. But since that fails, Snakemake protects you from a ‘bad run’ by deleting the resulting files. I since then use a workaround suggested by emma.

auspice is not ok either.
btw adding the path to auspice view command helped.

Kim

Glad things are working for you now!

Errors such as those you report installing auspice can be almost impossible to debug in this setting. If you want the latest auspice version, the easiest is probably to create a new conda environment and install into that (or delete the nextstrain environment and start over). If you wish to do that, here are the auspice install docs.

Hi James

I removed nextstrain and reinstalled it using conda

this the report and unfortunately auspice – version command gives the same error

(nextstrain) kimberleyb@DESKTOP-UC6PP0A:~$ nextstrain check-setup --set-default
nextstrain-cli is up to date!

Testing your setup…

docker is not supported

✘ no: docker is installed
✘ no: docker run works
? unknown: containers have access to >2 GiB of memory
:heavy_check_mark: yes: image is new enough for this CLI version

native is supported

:heavy_check_mark: yes: snakemake is installed
:heavy_check_mark: yes: augur is installed
:heavy_check_mark: yes: auspice is installed

aws-batch is not supported

✘ no: job description “nextstrain-job” exists
✘ no: job queue “nextstrain-job-queue” exists
✘ no: S3 bucket “nextstrain-jobs” exists

All good! Supported Nextstrain environments: native

Setting default environment to native.
(nextstrain) kimberleyb@DESKTOP-UC6PP0A:~$ auspice --version
/home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice: line 2: syntax error near unexpected token newline' /home/kimberleyb/miniconda3/envs/nextstrain/bin/auspice: line 2: Usage: npm ’

I guess I will have to work with the workaround for now. Maybe the next update will solve the problem

thanks for all the help
Kim

Hmmm, line 2: syntax error near unexpected token newline is something I haven’t seen before, and might be a Windows thing? Fingers crossed that the workaround can carry you through here.

the workaround works fine. Thanks for all the advise Really appreciate you guys having this forum.

Kim