Augur Refine Error

Help and Getting Started
1

Running a build with about ~2600 samples. I’ve run into the same error even after downloading the most recent ncov repository from Github. I’ve tried to trouble shoot this, but I’m at a stopping point. I tried do a --latency-wait 1000 and no change. I made a build with ~ 4K samples just before this so I’m not sure what/where to correct.

Any feedback would be appreciated.

MissingOutputException in line 689 of /nextstrain/build/workflow/snakemake_rules/main_workflow.smk:

Missing files after 300 seconds:

results/TGH/tree.nwk

results/TGH/branch_lengths.json

This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait.

Shutting down, this might take some time.

2

Hmm, could you check the output from logs/refine_TGH.txt to see if there’s any error printed from the “refine” step?

3

No error or message of any kind in the logs/refine_TGH.txt file

4

I think the original error was a resource usage error. I changed my Docker usage to --cores 4 and 4GB memory. It pass through, but now there is an error in augur --ancestral.

0.00 -TreeAnc: set-up

8.22 -SequenceData: loaded alignment.

8.23 -SeqData: making compressed alignment…

13.94 -SequenceData: constructed compressed alignment…
13.98 —WARNING: Number of tips in tree differs from number of sequences in
alignment!

14.03 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0575/2021

14.07 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0298/2021

14.12 ***WARNING: TreeAnc: 2 nodes don’t have a matching sequence in the
alignment. POSSIBLE ERROR.

15.14 -TreeAnc.infer_ancestral_sequences with method: probabilistic, joint
15.14 --TreeAnc._ml_anc_joint: type of reconstruction: Joint

0.00 -TreeAnc: set-up

2.73 -SequenceData: loaded alignment.

2.73 -SeqData: making compressed alignment…

4.76 -SequenceData: constructed compressed alignment…
4.79 —WARNING: Number of tips in tree differs from number of sequences in
alignment!

4.84 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0804/2021

4.87 ***WARNING: TreeAnc: 1 nodes don’t have a matching sequence in the
alignment. POSSIBLE ERROR.

5.35 -TreeAnc.infer_ancestral_sequences with method: probabilistic, joint
5.35 --TreeAnc._ml_anc_joint: type of reconstruction: Joint

0.00 -TreeAnc: set-up

1.89 -SequenceData: loaded alignment.

1.89 -SeqData: making compressed alignment…

3.12 -SequenceData: constructed compressed alignment…
3.14 —WARNING: Number of tips in tree differs from number of sequences in
alignment!

3.17 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0792/2021

3.25 ***WARNING: TreeAnc: 1 nodes don’t have a matching sequence in the
alignment. POSSIBLE ERROR.

3.54 -TreeAnc.infer_ancestral_sequences with method: probabilistic, joint
3.54 --TreeAnc._ml_anc_joint: type of reconstruction: Joint

0.00 -TreeAnc: set-up

1.29 -SequenceData: loaded alignment.

1.30 -SeqData: making compressed alignment…

1.61 -SequenceData: constructed compressed alignment…
1.64 —WARNING: Number of tips in tree differs from number of sequences in
alignment!

1.66 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-BPHL-4996/2021

1.69 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-BPHL-0832/2021

1.71 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/CA-LACPHL-AF00877/2021

1.74 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-BPHL-5006/2021

1.76 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0579/2021

1.79 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0580/2021

1.81 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0689/2021

1.84 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/TN-CDC-4157920-001/2021

1.86 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0756/2021

1.89 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0615/2021

1.91 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0687/2021

1.94 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0762/2021

1.97 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0768/2021

1.99 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/MN-MDH-8330/2021

2.02 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0653/2021

2.04 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0844/2021

2.07 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0784/2021

2.10 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0760/2021

2.13 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0771/2021

2.16 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0818/2021

2.19 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0139/2021

2.23 ***WARNING: TreeAnc._attach_sequences_to_nodes: NO SEQUENCE FOR LEAF:
USA/FL-TGH-0531/2021

2.27 ***WARNING: TreeAnc: 22 nodes don’t have a matching sequence in the
alignment. POSSIBLE ERROR.

3.85 -TreeAnc.infer_ancestral_sequences with method: probabilistic, joint
3.85 --TreeAnc._ml_anc_joint: type of reconstruction: Joint

0.00 -TreeAnc: set-up

1.13 -SequenceData: loaded alignment.

1.13 -SeqData: making compressed alignment…

1.29 -SequenceData: constructed compressed alignment…

1.45 -TreeAnc.infer_ancestral_sequences with method: probabilistic, joint
1.45 --TreeAnc._ml_anc_joint: type of reconstruction: Joint

0.00 -TreeAnc: set-up

1.24 -SequenceData: loaded alignment.

1.25 -SeqData: making compressed alignment…

1.58 -SequenceData: constructed compressed alignment…

1.77 -TreeAnc.infer_ancestral_sequences with method: probabilistic, joint
1.77 --TreeAnc._ml_anc_joint: type of reconstruction: Joint
[Tue Aug 10 20:11:52 2021]
Finished job 17.
19 of 31 steps (61%) done
Shutting down, this might take some time.
Exiting because a job execution failed. Look above for error message

I checked and all of the sequences it mentions were in the original data/sequences.fasta and proper data/metadata.tsv files. If samples are too closely related, would this account for the more intensive CPU usage and errors in tree building?