Test set Model Macro-AGM Macro-AGM SOV8 Q8
Residue level
Sequence level (±SD/p-value/d) Sequence level (±SD/p-value/d)
Residue level
TEST2018
ProteinUnetLM
0.728
0.728 ±0.182 0.760 ±0.144
0.756
NetSurfP-3.0
0.693
0.644 ±0.196/p=2e-5/d=0.443 0.695 ±0.171/p=2e-5/d=0.411
0.716
ProtT5Sec
0.800
0.688 ±0.193/p=2e-5/d=0.212 0.748 ±0.153/p=3e-4/d=0.080
0.749
SPOT-1D-LM
0.725
0.710 ±0.197/p=0.004/d=0.095 0.757 ±0.161/p=0.414/d=0.019
0.764
TEST2020
ProteinUnetLM
0.667
0.599 ±0.200 0.648 ±0.173
0.683
NetSurfP-3.0
0.718
0.509 ±0.204/p=2e-5/d=0.447 0.588 ±0.192/p=2e-5/d=0.325
0.662
ProtT5Sec
0.761
0.568 ±0.200/p=2e-5/d=0.156 0.638 ±0.179/p=0.002/d=0.055
0.681
SPOT-1D-LM
0.661
0.569 ±0.208/p=2e-5/d=0.149 0.643 ±0.181/p=0.140/d=0.028
0.693
TEST2020-HQ
ProteinUnetLM
0.678
0.623 ±0.192 0.685 ±0.167
0.689
NetSurfP-3.0
0.640
0.529 ±0.179/p=2e-5/d=0.501 0.625 ±0.178/p=2e-5/d=0.347
0.657
ProtT5Sec
0.668
0.595 ±0.186/p=5e-4/d=0.144 0.681 ±0.165/p=0.585/d=0.021
0.688
SPOT-1D-LM
0.674
0.601 ±0.189/p=0.008/d=0.111 0.687 ±0.165/p=0.761/d=-0.012
0.701
Neff1-2020
ProteinUnetLM
0.653
0.585 ±0.216 0.653 ±0.202
0.687
NetSurfP-3.0
0.635
0.506 ±0.229/p=2e-4/d=0.348 0.573 ±0.202/p=2e-4/d=0.391
0.678
ProtT5Sec
0.652
0.584 ±0.208/p=0.991/d=0.001 0.645 ±0.213/p=0.469/d=0.040
0.689
SPOT-1D-LM
0.647
0.574 ±0.227/p=0.368/d=0.049 0.652 ±0.213/p=0.900/d=0.007
0.701
CASP12-FM
ProteinUnetLM
0.663
0.609 ±0.161 0.650 ±0.114
0.651
NetSurfP-3.0
0.563
0.541 ±0.149/p=0.001/d=0.427 0.574 ±0.121/p=5e-4/d=0.631
0.610
ProtT5Sec
0.584
0.603 ± 0.155/p=0.736/d=0.035 0.633 ±0.109/p=0.059/d=0.150
0.640
SPOT-1D-LM
0.656
0.589 ±0.168/p=0.165/d=0.118 0.627 ±0.120/p=0.039/d=0.196
0.643