[SocBiN] Fwd: CASP: Call for targets for the 2024 CASP modeling experiment

[Arne Elofsson]

CASP (Critical Assessment of Structure Prediction) experiments are held
every two years. Recent rounds have seen dramatic increases in modeling
accuracy, resulting from the introduction of deep learning methods: In
2018, for the first time, the folds of most proteins were correctly
computed1; in 2020, the accuracy of many computed protein structures
rivaled that of the corresponding experimental ones2; in 2022, there was an
enormous increase in the accuracy of protein complexes3.

We have seen the beginning of what deep learning methods may achieve in
structural biology. In addition to further increases in the accuracy of
protein complexes, methods are being developed for RNA structures, organic
ligand-protein complexes, and for moving beyond single macromolecular
structures to compute conformational ensembles. Accurate computational
methods together with experimental data also offer the prospect of probing
previously inaccessible biological systems. CASP has expanded its scope to
provide critical assessment in all these areas.

CASP is only possible with the generous participation of the experimental
structural biology community in providing suitable targets: A total of over
1100 targets have been obtained over the previous CASP rounds. We are now
requesting targets for the 2024 CASP16 experiment. We need challenge
targets in the following areas:

Single protein structures: The 2020 and 2022 CASPs showed that, so far,
Alphafold2 and methods built around it are by far the most accurate4. But
there are limitations, particularly for some proteins where only a shallow
sequence alignment is available and for very large proteins (more than 1000
amino acids). The best results also require substantial amounts of
computing resources, well beyond that of the AlphaFold2 default settings.
Many new methods are continuing to appear and these may remove some of the
remaining difficulties. All types of protein targets are needed, but
especially those with shallow sequence alignments, without structural
templates, and large proteins.

Protein complexes: In the 2022 CASP15, advanced deep learning methods were
applied to protein complexes for the first time5. The result was a huge
improvement in accuracy compared with classical docking approaches. But
overall, the results are still not at the level achieved for single
proteins. So, in CASP16 we need all sorts of targets in this area so as to
determine progress since then. We particularly need complexes where there
is no evolutionary information across the protein-protein interfaces, for
example, antibody-antigen complexes. (This CASP category is conducted in
close collaboration with our colleagues at CAPRI - Critical Assessment of
protein interactions6).

Nucleic acid structures and complexes: In recognition of the major role
nucleic acid structures and complexes play in biology, CASP now includes
this class of target. A number of papers claiming successful RNA structure
computation using deep learning methods have been published, but those
participating in the 2022 CASP RNA category performed less well than
classical approaches, and no methods were able to effectively address the
two RNA protein-complexes included7. CASP needs a wide variety of RNA, DNA,
and complexes as targets to see if this situation has changed. (This CASP
category is conducted in close collaboration with RNApuzzles8).

Organic ligand-protein complexes: This area is of major importance for
computer-aided drug discovery. Earlier, there have been community
experiments to assess the accuracy of methods, particularly SAMPL, CSAR,
D3R, and a new one, CACHE, has recently started (cache-challenge.org).
These challenges have drawn strong international participation from
researchers in both academia and industry. Here too, a number of promising
deep learning papers have appeared, but in the 2022 CASP15 pilot, classical
methods were still superior9. So, we need appropriate targets to see if
progress has been made since. Ideally, these should be sets of
three-dimensional protein-ligand complexes from drug discovery projects,
but single targets would also be appreciated. Additionally, where
available, we will assess non-structural quantities such as affinities or
affinity rankings and other properties of pharmaceutical interest when
these are available (small molecule pKs, and DMPK related properties).

Ensembles of macromolecule conformations: It is now widely recognized that
proteins and nucleic acids often adopt multiple conformations that can
underpin their functions. In these cases, considering only a single protein
or RNA conformation may be a significant oversimplification. The 2022
CASP15 included a pilot experiment to assess methods for computing multiple
conformations, with encouraging results10, but with limitations imposed by
the available experimental data. For 2024, we seek not only cases of
multiple experimental three-dimensional structures for the same
macromolecule but also other types of data that might be used for
assessment of computed conformation ensembles such as cryoEM, NMR, X-ray
crystallography, SAXS, and/or cross-link data.

Integrative modeling: The more powerful computational methods open up new
possibilities for combination with sparse or low-resolution experimental
data to investigate previously inaccessible biological structures and
machines. CASP is interested in exploring these possibilities and so
requests experimentally difficult targets where structure has nevertheless
been obtained. In appropriate cases, we expect to be able to collaborate
with other experimental groups to provide appropriate data from NMR,
cross-linking or SAXS.

There are three avenues to contribute a target to CASP:
1.      (preferrable) Submit directly to CASP through our web-interface
https://predictioncenter.org/casp16/targets_submission_form.cgi (requires a
quick registration at https://predictioncenter.org/login.cgi if you do not
have an account with us).
2.      Email to targets at predictioncenter.org with your target suggestions
or to discuss any questions.
3.      Submit your structure to the PDB (on-hold) and designate it as a
CASP target through PDB’s submission interface.

The timeline for the 2024 CASP requires that targets are submitted starting
now and until July 1. We would like to hear from you as soon as possible if
you may have something suitable or have suggestions about other target
sources. In order to maintain rigor, the experimental data for a target
must not be publicly available until after computed structures have been
collected. For assessment, CASP requires the experimental data by August
15, but the data can remain confidential after that.  Target providers are
invited to contribute to papers11-15 for a special CASP issue of the
journal Proteins.


1.      Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical
assessment of methods of protein structure prediction (CASP)-Round XIII.
Proteins 2019;87(12):1011-1020.
2.      Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical
assessment of methods of protein structure prediction (CASP)-Round XIV.
Proteins 2021;89(12):1607-1617.
3.      Kryshtafovych A, Schwede T, Topf M, Fidelis K, Moult J. Critical
assessment of methods of protein structure prediction (CASP)-Round XV.
Proteins 2023;91(12):1539-1549.
4.      Simpkin AJ, Mesdaghi S, Sanchez Rodriguez F, Elliott L, Murphy DL,
Kryshtafovych A, Keegan RM, Rigden DJ. Tertiary structure assessment at
CASP15. Proteins 2023;91(12):1616-1635.
5.      Ozden B, Kryshtafovych A, Karaca E. The impact of AI-based modeling
on the accuracy of protein assembly prediction: Insights from CASP15.
Proteins 2023;91(12):1636-1657.
6.      Lensink MF, Brysbaert G, Raouraoua N, Bates PA, Giulini M, Honorato
RV, van Noort C, Teixeira JMC, Bonvin A, Kong R, Shi H, Lu X, Chang S, Liu
J, Guo Z, Chen X, Morehead A, Roy RS, Wu T, Giri N, Quadir F, Chen C, Cheng
J, Del Carpio CA, Ichiishi E, Rodriguez-Lumbreras LA, Fernandez-Recio J,
Harmalkar A, Chu LS, Canner S, Smanta R, Gray JJ, Li H, Lin P, He J, Tao H,
Huang SY, Roel-Touris J, Jimenez-Garcia B, Christoffer CW, Jain AJ, Kagaya
Y, Kannan H, Nakamura T, Terashi G, Verburgt JC, Zhang Y, Zhang Z, Fujuta
H, Sekijima M, Kihara D, Khan O, Kotelnikov S, Ghani U, Padhorny D, Beglov
D, Vajda S, Kozakov D, Negi SS, Ricciardelli T, Barradas-Bautista D, Cao Z,
Chawla M, Cavallo L, Oliva R, Yin R, Cheung M, Guest JD, Lee J, Pierce BG,
Shor B, Cohen T, Halfon M, Schneidman-Duhovny D, Zhu S, Yin R, Sun Y, Shen
Y, Maszota-Zieleniak M, Bojarski KK, Lubecka EA, Marcisz M, Danielsson A,
Dziadek L, Gaardlos M, Gieldon A, Liwo A, Samsonov SA, Slusarz R, Zieba K,
Sieradzan AK, Czaplewski C,
 Kobayashi S, Miyakawa Y, Kiyota Y, Takeda-Shitaka M, Olechnovic K,
Valancauskas L, Dapkunas J, Venclovas C, Wallner B, Yang L, Hou C, He X,
Guo S, Jiang S, Ma X, Duan R, Qui L, Xu X, Zou X, Velankar S, Wodak SJ.
Impact of AlphaFold on structure prediction of protein complexes: The
CASP15-CAPRI experiment. Proteins 2023;91(12):1658-1683.
7.      Das R, Kretsch RC, Simpkin AJ, Mulvaney T, Pham P, Rangan R, Bu F,
Keegan RM, Topf M, Rigden DJ, Miao Z, Westhof E. Assessment of
three-dimensional RNA structure prediction in CASP15. Proteins
2023;91(12):1747-1770.
8.      Magnus M, Antczak M, Zok T, Wiedemann J, Lukasiak P, Cao Y,
Bujnicki JM, Westhof E, Szachniuk M, Miao Z. RNA-Puzzles toolkit: a
computational resource of RNA 3D structure benchmark datasets, structure
manipulation, and evaluation tools. Nucleic Acids Res 2020;48(2):576-588.
9.      Robin X, Studer G, Durairaj J, Eberhardt J, Schwede T, Walters WP.
Assessment of protein-ligand complexes in CASP15. Proteins
2023;91(12):1811-1821.
10.     Kryshtafovych A, Montelione GT, Rigden DJ, Mesdaghi S, Karaca E,
Moult J. Breaking the conformational ensemble barrier: Ensemble structure
modeling challenges in CASP15. Proteins 2023;91(12):1903-1911.
11.     Kretsch RC, Andersen ES, Bujnicki JM, Chiu W, Das R, Luo B,
Masquida B, McRae EKS, Schroeder GM, Su Z, Wedekind JE, Xu L, Zhang K,
Zheludev IN, Moult J, Kryshtafovych A. RNA target highlights in CASP15:
Evaluation of predicted models by structure providers. Proteins
2023;91(12):1600-1615.
12.     Alexander LT, Durairaj J, Kryshtafovych A, Abriata LA, Bayo Y,
Bhabha G, Breyton C, Caulton SG, Chen J, Degroux S, Ekiert DC, Erlandsen
BS, Freddolino PL, Gilzer D, Greening C, Grimes JM, Grinter R, Gurusaran M,
Hartmann MD, Hitchman CJ, Keown JR, Kropp A, Kursula P, Lovering AL,
Lemaitre B, Lia A, Liu S, Logotheti M, Lu S, Markusson S, Miller MD,
Minasov G, Niemann HH, Opazo F, Phillips GN, Jr., Davies OR, Rommelaere S,
Rosas-Lemus M, Roversi P, Satchell K, Smith N, Wilson MA, Wu KL, Xia X,
Xiao H, Zhang W, Zhou ZH, Fidelis K, Topf M, Moult J, Schwede T. Protein
target highlights in CASP15: Analysis of models by structure providers.
Proteins 2023;91(12):1571-1599.
13.     Alexander LT, Lepore R, Kryshtafovych A, Adamopoulos A, Alahuhta M,
Arvin AM, Bomble YJ, Bottcher B, Breyton C, Chiarini V, Chinnam NB, Chiu W,
Fidelis K, Grinter R, Gupta GD, Hartmann MD, Hayes CS, Heidebrecht T, Ilari
A, Joachimiak A, Kim Y, Linares R, Lovering AL, Lunin VV, Lupas AN, Makbul
C, Michalska K, Moult J, Mukherjee PK, Nutt WS, Oliver SL, Perrakis A,
Stols L, Tainer JA, Topf M, Tsutakawa SE, Valdivia-Delgado M, Schwede T.
Target highlights in CASP14: Analysis of models by structure providers.
Proteins 2021;89(12):1647-1672.
14.     Lepore R, Kryshtafovych A, Alahuhta M, Veraszto HA, Bomble YJ,
Bufton JC, Bullock AN, Caba C, Cao H, Davies OR, Desfosses A, Dunne M,
Fidelis K, Goulding CW, Gurusaran M, Gutsche I, Harding CJ, Hartmann MD,
Hayes CS, Joachimiak A, Leiman PG, Loppnau P, Lovering AL, Lunin VV,
Michalska K, Mir-Sanchis I, Mitra AK, Moult J, Phillips GN, Jr., Pinkas DM,
Rice PA, Tong Y, Topf M, Walton JD, Schwede T. Target highlights in CASP13:
Experimental target structures through the eyes of their authors. Proteins
2019;87(12):1037-1057.
15.     Kryshtafovych A, Albrecht R, Basle A, Bule P, Caputo AT, Carvalho
AL, Chao KL, Diskin R, Fidelis K, Fontes C, Fredslund F, Gilbert HJ,
Goulding CW, Hartmann MD, Hayes CS, Herzberg O, Hill JC, Joachimiak A,
Kohring GW, Koning RI, Lo Leggio L, Mangiagalli M, Michalska K, Moult J,
Najmudin S, Nardini M, Nardone V, Ndeh D, Nguyen TH, Pintacuda G, Postel S,
van Raaij MJ, Roversi P, Shimon A, Singh AK, Sundberg EJ, Tars K, Zitzmann
N, Schwede T. Target highlights from the first post-PSI CASP experiment
(CASP12, May-August 2016). Proteins 2018;86 Suppl 1(Suppl 1):27-50.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.su.se/pipermail/socbin-at-sbc.su.se/attachments/20240318/a2be2ec5/attachment.html>