I’m getting the error “Failed to fetch frequencies” when I run the nextstrain server. I have a _tip-frequencies.json file in the same folder as the main JSON file and can see that the server logs are catching the file. Upon opening the console logs in Inspect mode, I get the error Failed to fetch frequencies e.values is undefined. How can I debug this error?
Hi @syaffers – it looks like auspice is unable to parse the tip-frequencies file. While we don’t have a schema for that file type, you could try comparing your file contetnts to a file which is working, such as influenza / seasonal / yam / na / 2y.
If that’s not helpful, you can run auspice develop ... instead of auspice view ... which should give you a more helpful error message in the console which may help us debug further.
Ah, I think I see the issue. My _tip-frequencies file contain the key global instead of frequencies as in the examples that you showed. This is generated through the following Snakefile rule:
rule frequencies:
message: "Generating frequencies plot"
input:
tree = rules.refine.output.tree,
metadata = input_metadata
params:
method = "diffusion",
stiffness = 3.0,
min_date = "2020-03-01",
max_date = "2021-02-14",
pivot_interval = 1,
pivot_interval_units = "weeks"
output:
frequencies = "auspice/qc10Feb_blastMar3_k30_mm15_tip-frequencies.json"
shell:
"""
augur frequencies \
--tree {input.tree} \
--metadata {input.metadata} \
--method {params.method} \
--stiffness {params.stiffness} \
--max-date {params.max_date} \
--min-date {params.min_date} \
--pivot-interval {params.pivot_interval} \
--pivot-interval-units {params.pivot_interval_units} \
--output {output.frequencies}
"""
Is this the correct way to generate the frequencies panel?
That looks similar to the rule we use for ncov, but i’ll tag @jlhudd as he will know more.
1 Like
Changing the method to kde generated the frequencies key in the JSON. Now I can view the frequencies panel in the auspice server. Just for my reference, I’ve got a few questions:
- Is this the expected way to do it?
- What the difference between KDE and diffusion?
- Also, what’s the
globalkey for if we set the method todiffusion?
Hi @syaffers!
Is this the expected way to do it?
Yes, this is the standard way we calculate tip frequencies for most of our pathogen analyses.
What the difference between KDE and diffusion?
The short answer is that KDE estimation uses a “bottom-up” approach assigning frequencies to tips in the tree and summing these to calculate internal node frequencies while diffusion estimation uses a “top-down” approach assigning 100% frequency to the root node of the tree and partitioning this total frequency to calculate internal node frequencies down to the tips.
For an example of how KDE frequencies work, check out our draft guide of how to calculate your own clade frequencies for SARS-CoV-2. We don’t have a similar guide for diffusion frequencies yet.
Also, what’s the
globalkey for if we set the method todiffusion?
The diffusion frequencies were originally developed to work with an older version of Auspice that expected frequencies to be reported in a different data structure (i.e., grouped by region). These frequencies and the corresponding data structure are still useful when we analyze regional clade frequencies for seasonal influenza, but we don’t use diffusion frequencies as much elsewhere.
1 Like
@jlhudd hi I’ ve got some trouble with my auspice—on my auspice.us page, "show frequencies"panel exists, however, I cannot view my frequencies’ outcome. what’s wrong with it?
I used the zika tutorial snakefile as a template for my local builds.
I’ve added “frequencies” to the panels in the JSON and file: “tip-frequencies.json” is same as the main dataset json , e.g: auspice/new-zika-build_2022.json auspice/new-zika-build_2022_tip-frequencies.json
Thanks for advance!
file auspice/new-zika-build_2022.json:
{
"generated_by": {
"program": "augur",
"version": "13.1.0"
},
"nodes": {
"1_0087_PF": {
"frequencies": [
0.169878,
0.41524,
0.382693,
0.34689,
0.016553,
4.1e-05,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
]
},
"1_0181_PF": {
"frequencies": [
0.169878,
0.41524,
0.382693,
0.34689,
0.016553,
4.1e-05,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
]
},
"1_0199_PF": {
"frequencies": [
0.660244,
0.16952,
0.23338,
0.158837,
0.005426,
1e-05,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0
]
},
"Aedes_aegypti/USA/2016/FL05": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
5.1e-05,
0.000227,
0.002419,
0.028807,
0.128576,
0.096361
]
},
"BRA/2016/FC_6706": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
1.4e-05,
0.000874,
0.006931,
0.006402,
0.144304,
0.015968,
0.001978,
0.002826
]
},
"Brazil/2015/ZBRA105": {
"frequencies": [
0.0,
1e-06,
0.001164,
0.128854,
0.715467,
0.829044,
0.363754,
0.036698,
0.003251,
3e-05,
1e-06,
0.0,
0.0,
0.0
]
},
"Brazil/2015/ZBRC301": {
"frequencies": [
0.0,
0.0,
3.6e-05,
0.009439,
0.124501,
0.085263,
0.319575,
0.393956,
0.018078,
0.000398,
2.1e-05,
1e-06,
0.0,
0.0
]
},
"Brazil/2015/ZBRC303": {
"frequencies": [
0.0,
0.0,
3.4e-05,
0.009088,
0.121193,
0.083886,
0.307459,
0.411421,
0.018388,
0.00041,
2.2e-05,
1e-06,
0.0,
0.0
]
},
"Brazil/2016/ZBRC16": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
3e-06,
3.6e-05,
0.000333,
0.008786,
0.030134,
0.112222,
0.016059,
0.003481,
0.000251,
0.00015
]
},
"COL/FLR_00008/2015": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
2.2e-05,
0.000159,
0.001024,
0.018602,
0.074731,
0.117965,
0.006027,
0.001664,
8.3e-05,
3.4e-05
]
},
"COL/FLR_00024/2015": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
2.2e-05,
0.000159,
0.001024,
0.018602,
0.074731,
0.117965,
0.006027,
0.001664,
8.3e-05,
3.4e-05
]
},
"Colombia/2016/ZC204Se": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
7e-06,
6.4e-05,
0.00052,
0.011893,
0.038694,
0.131069,
0.009877,
0.002669,
0.000167,
8.6e-05
]
},
"DOM/2016/BB_0059": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
1.7e-05,
0.000999,
0.007582,
0.007016,
0.14722,
0.014098,
0.001817,
0.002485
]
},
"DOM/2016/BB_0183": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
9e-06,
0.000621,
0.005491,
0.005287,
0.128051,
0.023224,
0.002426,
0.003867
]
},
"DOM/2016/BB_0433": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
7.3e-05,
0.001195,
0.002034,
0.016059,
0.1921,
0.006332,
0.017948
]
},
"DOM/2016/MA_WGS16_011": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
9.8e-05,
0.001476,
0.002327,
0.021903,
0.165098,
0.005649,
0.015121
]
},
"EcEs062_16": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1e-05,
0.000677,
0.005827,
0.005525,
0.133156,
0.020982,
0.002308,
0.003579
]
},
"HND/2016/HU_ME59": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
3e-06,
0.00025,
0.002911,
0.003539,
0.063131,
0.071475,
0.00384,
0.008036
]
},
"Nica1_16": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
3e-06,
3.6e-05,
0.000333,
0.008786,
0.030134,
0.112222,
0.016059,
0.003481,
0.000251,
0.00015
]
},
"PAN/CDC_259359_V1_V3/2015": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
1.9e-05,
0.000146,
0.000962,
0.017866,
0.069015,
0.121435,
0.006249,
0.001745,
8.9e-05,
3.7e-05
]
},
"PRVABC59": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
2.2e-05,
0.000159,
0.001024,
0.018602,
0.074731,
0.117965,
0.006027,
0.001664,
8.3e-05,
3.4e-05
]
},
"SG_018": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
4.3e-05,
0.000201,
0.002238,
0.024217,
0.139143,
0.102975
]
},
"SG_027": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2e-06,
8.6e-05,
0.000333,
0.003076,
0.05229,
0.089821,
0.07844
]
},
"SG_056": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2e-06,
8.3e-05,
0.000324,
0.003022,
0.049944,
0.092821,
0.079687
]
},
"SG_074": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2e-06,
8.3e-05,
0.000324,
0.003022,
0.049944,
0.092821,
0.079687
]
},
"SMGC_1": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
1e-06,
1e-05,
0.000129,
0.004513,
0.019867,
0.051465,
0.051611,
0.00558,
0.000534,
0.000423
]
},
"Thailand/1610acTw": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1e-05,
6.9e-05,
0.001094,
0.009374,
0.146146,
0.247749
]
},
"USA/2016/FL022": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
1.3e-05,
0.000331,
0.000863,
0.005539,
0.184838,
0.019355,
0.041901
]
},
"USA/2016/FLUR022": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2e-06,
7.4e-05,
0.000297,
0.002863,
0.043482,
0.101918,
0.083536
]
},
"USA/2016/FLWB042": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
0.0,
2.5e-05,
0.000134,
0.001715,
0.014976,
0.161473,
0.132609
]
},
"V8375": {
"frequencies": [
0.0,
0.0,
0.0,
2e-06,
0.000207,
0.000932,
0.003662,
0.041,
0.487419,
0.025207,
0.002854,
0.000505,
1.5e-05,
4e-06
]
},
"VEN/UF_1/2016": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
0.0,
1e-06,
2.6e-05,
0.001383,
0.009412,
0.009328,
0.144304,
0.010959,
0.001456,
0.001786
]
},
"ZKC2/2016": {
"frequencies": [
0.0,
0.0,
0.0,
0.0,
1e-06,
9e-06,
0.000119,
0.004273,
0.01922,
0.047418,
0.056049,
0.005768,
0.000564,
0.000457
]
},
"generated_by": {
"program": "augur",
"version": "13.1.0"
}
},
"pivots": [
2013.75,
2014.0,
2014.25,
2014.5,
2014.75,
2015.0,
2015.25,
2015.5,
2015.75,
2016.0,
2016.25,
2016.5,
2016.75,
2017.0
]
}
I found that I couldnt see the attribute—frequency in each tips
file auspice/new-zika-build_2022_tip-frequencies.json:
"name": "Thailand/1610acTw",
"node_attrs": {
"accession": "MF692778",
"author": {
"author": "Lin et al",
"journal": "Unpublished",
"paper_url": "https://www.ncbi.nlm.nih.gov/pubmed/",
"title": "Imported Zika virus strains, Taiwan, 2016",
"value": "Lin et al"
},
"country": {
"confidence": {
"Thailand": 1.0
},
"entropy": -1.000088900581841e-12,
"value": "Thailand"
},
"div": 0.006057546290449948,
"num_date": {
"confidence": [
2016.7596394392926,
2016.8319672131147
],
"value": 2016.8319672131147
},
"region": {
"confidence": {
"Southeast Asia": 1.0
},
"entropy": -1.000088900581841e-12,
"value": "Southeast Asia"
},
"url": "https://www.ncbi.nlm.nih.gov/nuccore/MF692778"
}
},
{
"branch_attrs": {
"labels": {
"aa": "NS1: V194A; PRO: T74A"
},
"mutations": {
"NS1": [
"V194A"
],
"PRO": [
"T74A"
],
"nuc": [
"A676G",
"C1221T",
"T1848C",
"T3053C",
"A3360G",
"T3928C",
"G4152A",
"A5727G",
"T6087C"
]
}
},
"children": [
{
"branch_attrs": {
"labels": {
"aa": "PRO: G79V"
},
"mutations": {
"PRO": [
"G79V"
],
"nuc": [
"G692T",
"C4716T",
"G4767A",
"G6489A",
"A7821G",
"A10335G"
]
}
},
@Bayley - auspice.us cannot currently parse sidecar files (such as _tip-frequencies.json) but this functionality should be there shortly.
Hope this functionality comes out soon.
thanks for your reply!
Should be working now in auspice.us (version 0.11). Let us know if there are any issues!
I tried uploading mine zika data in auspice.us, but there nothing changed, I cant view frequency fig as same as before.
Is there something new with auspice.us in version 0.11 ?
Perhaps I havent updated auspice.us, but I dont know how to update it.
So could frequency functionality be applid in other virus or bacterium (it only used in ncov and flu)?
Thanks for advance!
Hope there’re some tutorials with augur subcommand—frequency, like other subcommands
Oh! I get it, 
I forgot using “$ next view auspice” command in env(nextstrain), I used to put XXX.json in auspice.us so that there cant appear frequency fig.
Thanks for @james 's help, maybe the version 0.11 of auspice.us plays a part in solving this issue.
And now I could use frequency functionality with other pathogen.
Nice day!
