CASP Data Processing

This repository contains scripts and utilities for processing, analyzing, and visualizing data for the CASP (Critical Assessment of Structure Prediction) assessment. The workflow includes processing scores, identifying missing data points, and generating plots and summary files.

This work is used in the following publication:

Dube, N., Ramelot, T. A., Benavides, T. L., Huang, Y. J., Moult, J., Kryshtafovych, A., & Montelione, G. T. (2025). Modeling Alternative Conformational States in CASP16. bioRxiv. https://doi.org/10.1101/2025.09.02.673835

Directory Structure

CASP_DATA_PROCESSING/
├── data/                 # Input CSV data files
│   ├── group_number_name_correspondance.csv  # Group number to name mapping
├── output/               # All generated outputs (CSVs, plots, tables, PDFs)
│   ├── OUTPUT_CSVS/      # Output CSVs and summary files
│   ├── PLOTS/            # Generated plots and visualizations
│   ├── PLOTS_MANUSCRIPT/ # Publication-ready plots for manuscript figures
│   ├── LATEX_TABLES/     # LaTeX table files for publication
│   ├── DUAL_STATE_LATEX_TABLES/   # LaTeX tables for dual-state scores
│   ├── SINGLE_STATE_LATEX_TABLES/ # LaTeX tables for single-state scores
│   └── CASP16_*.pdf      # PDF reports generated by scripts
├── scripts/              # Processing and plotting scripts
│   ├── process_two_state_score.py
│   ├── process_two_state_score_full_axis.py
│   ├── process_two_state_score_simple_bar_plots.py
│   ├── process_TM_GDT_two_state_multipanel.py
│   ├── process_single_state_score.py
│   ├── process_dual_state_score.py
│   ├── create_latex_tables.py
│   ├── create_dual_state_latex_tables.py
│   ├── create_single_state_latex_tables.py
│   └── MakePlotsForManuscript.py
├── archive/              # Archived plots, PDFs, and legacy pipeline
│   ├── archive/
│   │   ├── PLOTS/
├── original_pipeline_by_NamitaDube_2024/ # Legacy modular pipeline (non-functional)
├── requirements.txt      # Python package dependencies
└── README.md

Prerequisites

Python 3.7+
Required Python packages (see requirements.txt for complete list)

Installation

Option 1: Using requirements.txt (Recommended)

Install all dependencies using the provided requirements file:

pip install -r requirements.txt

Option 2: Manual Installation

Install dependencies individually:

pip install pandas matplotlib adjustText tqdm

Option 3: Using a Virtual Environment (Recommended for Development)

Create and activate a virtual environment to avoid conflicts:

# Create virtual environment
python -m venv casp_env

# Activate virtual environment
# On Windows:
casp_env\Scripts\activate
# On macOS/Linux:
source casp_env/bin/activate

# Install dependencies
pip install -r requirements.txt

# Deactivate when done
deactivate

Quick Start

Clone or download this repository
Install dependencies:
```
pip install -r requirements.txt
```
Ensure your data files are in the data/ directory

Run the main processing script:

python scripts/process_two_state_score.py

Check the output/ directory for generated plots, CSVs, and tables

For a complete workflow, see the Complete Workflow section below.

Script Overview and Usage

1. `process_two_state_score.py`

Purpose: Processes the gathered data to compute two-state scores for each target and score type, generates summary CSVs in output/OUTPUT_CSVS/, and creates scatter and bar plots in output/PLOTS/.

Usage:

python scripts/process_two_state_score.py

What it does:

Automatically processes all targets and score types defined in TARGET_SCORE_DICT
Computes two-state scores by combining the best scores from different model/version combinations
Generates scatter plots comparing scores from V1 and V2 reference states
Creates stacked bar charts showing combined scores
Saves detailed CSV outputs with all scoring information
Handles special cases for different targets (M1228, M1239, R1203, T1214, T1228, T1239, T1249)

Targets and Score Types Processed:

M1228: BestDockQ, GDT_TS, GlobDockQ, GlobalLDDT, TMscore
M1239: BestDockQ, GDT_TS, GlobDockQ, GlobalLDDT, TMscore
R1203: GDT_TS, GlobalLDDT, Composite_Score_4, TMscore
T1214: GDT_TS, GlobalLDDT, TMscore, Composite_Score_4
T1228: GDT_TS, GlobalLDDT, TMscore
T1239: GDT_TS, GlobalLDDT, TMscore
T1249: AvgDockQ, GlobalLDDT, GDT_TS, TMscore

Algorithm 1 This algorithm describes the process for calculating two-state scores in this work based on scores calculated from two predicted states to two reference states.

2. `process_two_state_score_full_axis.py`

Purpose: Processes two-state scores plots with x and y axis ranges (0,1)

Usage:

python scripts/process_two_state_score_full_axis.py

3. `process_two_state_score_simple_bar_plots.py`

Purpose: Processes two-state scores with simplified bar plot visualizations, focusing on clean and minimal design.

Usage:

python scripts/process_two_state_score_simple_bar_plots.py

What it does:

Creates simplified bar plots with minimal visual elements
Focuses on clean, publication-ready visualizations
Reduces visual clutter while maintaining data clarity
Generates both horizontal and vertical bar plots
Handles the same targets and score types as the standard processor

4. `process_TM_GDT_two_state_multipanel.py`

Purpose: Creates multi-panel plots specifically for TM-score and GDT_TS scores, providing comprehensive visualization of these key metrics.

Usage:

python scripts/process_TM_GDT_two_state_multipanel.py

What it does:

Creates multi-panel plots combining TM-score and GDT_TS score visualizations
Provides comprehensive view of these critical assessment metrics
Generates publication-ready multi-panel figures
Combines scatter plots and bar charts in unified layouts
Focuses on TM-score and GDT_TS scores across all relevant targets

Figure S11. Two-State TM and GDT_TS scores for six targets: M1228, M1239, T1228, T1239, T1249, R1203.

5. `process_single_state_score.py`

Purpose: Processes single-state scores for targets that only have one reference state (currently T1214).

Usage:

python scripts/process_single_state_score.py

What it does:

Processes targets that don't have V1/V2 reference state comparisons
Generates single-state scoring analysis
Creates appropriate visualizations for single-state data
Currently configured for T1214 with GDT_TS, GlobalLDDT, TMscore, and Composite_Score_4

6. `create_latex_tables.py`

Purpose: Generates LaTeX table files from the processed CSV outputs for publication purposes.

Usage:

python scripts/create_latex_tables.py

What it does:

Reads all CSV files from the output/OUTPUT_CSVS/ directory
Converts them to properly formatted LaTeX tables
Creates a special T1214 Sigma4 score table
Saves all tables to the LATEX_TABLES/ directory

7. `process_dual_state_score.py`

Purpose: Processes dual-state scores for targets that have both V1 and V2 reference states but use a different scoring methodology than the standard two-state approach.

Usage:

python scripts/process_dual_state_score.py

What it does:

Processes dual-state scoring for T1214 target with multiple score types
Handles GDT_TS, GlobalLDDT, TMscore, and Composite_Score_1-4 for T1214

T1214 GDT_TS Dual State Scatterplot Dual state plots are similar to Two state plots, except that they use only a single model from each group and compare the best score to V1 (x-axis) vs V2 (y-axis). It is used in cases where predictors were only trying to model a single state V1, as in T1214. The dual state scatterplot allows for visualization of how a group’s best scoring model (vs V1) scores vs the original state V2.

Targets and Score Types Processed:

T1214: GDT_TS, GlobalLDDT, TMscore, Composite_Score_1, Composite_Score_2, Composite_Score_3, Composite_Score_4

8. `create_dual_state_latex_tables.py`

Purpose: Generates LaTeX table files specifically for dual-state score outputs, providing publication-ready tables with proper formatting.

Usage:

python scripts/create_dual_state_latex_tables.py

What it does:

Reads dual-state CSV files from the output/OUTPUT_CSVS/ directory
Converts them to properly formatted LaTeX tables
Saves all tables to the DUAL_STATE_LATEX_TABLES/ directory

9. `create_single_state_latex_tables.py`

Purpose: Generates LaTeX table files specifically for single-state score outputs, including special T1214 Sigma score tables.

Usage:

python scripts/create_single_state_latex_tables.py

What it does:

Reads single-state CSV files from the output/OUTPUT_CSVS/ directory
Creates special T1214 Sigma score tables for Composite_Score_1-4
Saves all tables to the SINGLE_STATE_LATEX_TABLES/ directory

10. `MakePlotsForManuscript.py`

Purpose: Generates publication-ready plots specifically for manuscript figures, with consistent formatting and styling.

Usage:

python scripts/MakePlotsForManuscript.py

What it does:

Creates publication-ready plots for specific manuscript figures
Generates Figure 1C, 2B-C, 4C, 5C, 6B-C, 7C-D and supplementary figures
Uses consistent formatting, fonts, and styling for publication
Saves all plots to the PLOTS_MANUSCRIPT/ directory

11. `original_pipeline_by_NamitaDube_2024/` (Legacy Directory)

Purpose: Contains the original modular pipeline for computing Composite Score 4 (CS4) across predicted protein structures. This is legacy code that does not function in the current environment but served as the foundation for developing the current repository's scripts.

Author: Namita Dube (2024)
Status: Non-functional legacy code, preserved for reference

What it contains:

Modular pipeline design: Extract → merge → normalize → score → plot workflow
10 Python scripts for processing LGA outputs, GlobalLDDT logs, and local GDT/LDDT tables
Makefile for automated pipeline execution
Composite Score 4 calculation: Mean of 5 components (Local_LDDT, Local_GDT, binary thresholds for Global_LDDT>80, Global_GDT>80, and g_RMS)
Configurable thresholds for Global_GDT_TS and Global_LDDT (default 80)
Plotting capabilities for bar plots and scatter diagnostic plots

Key Scripts:

extract_global_gdt_rms.py - Parse Global GDT_TS and g_RMS from LGA outputs
extract_global_lddt.py - Parse Global LDDT from .lddt outputs
extract_local_gdt.py - Compute Local GDT_TS averages for residue ranges
extract_local_lddt.py - Compute Local LDDT averages for residue ranges
merge_all.py - Merge metrics into combined_results.csv
normalize_metrics.py - Apply sigmoid scaling and ×100 scaling
compositescores.py - Compute Composite_Score_4 (CS1-CS4)
plot_models_bar.py - Generate bar plots
plot_best_per_group.py - Plot best scores per group
plot_scatter_diagnostic.py - Generate scatter diagnostic plots

Note: This directory is preserved for historical reference and to understand the evolution of the current processing pipeline.

Complete Workflow

Process Two-State Scores:

python scripts/process_two_state_score.py

Process Two-State Scores with Full Axis:

python scripts/process_two_state_score_full_axis.py

Process Two-State Scores with Simple Bar Plots:

python scripts/process_two_state_score_simple_bar_plots.py

Create Multi-Panel TM/GDT Plots:

python scripts/process_TM_GDT_two_state_multipanel.py

Process Single-State Scores (if applicable):

python scripts/process_single_state_score.py

Process Dual-State Scores (if applicable):

python scripts/process_dual_state_score.py

Generate LaTeX Tables:
```
python scripts/create_latex_tables.py
```

Generate Dual-State LaTeX Tables:

python scripts/create_dual_state_latex_tables.py

Generate Single-State LaTeX Tables:

python scripts/create_single_state_latex_tables.py

Generate Manuscript Plots:

python scripts/MakePlotsForManuscript.py

Output Files

CSV Outputs (`output/OUTPUT_CSVS/` directory)

Each output CSV file contains detailed scoring information with columns:

Group: The submission group identifier (e.g., TS314)
Group_Name: Human-readable group name
Combined_Score: The sum of the two best scores for the group
Best_v1_ref: The best score using a v1 reference
Best_v2_ref: The best score using a v2 reference
V1_Model_For_Combined_Score: Model identifier for the V1 score
V2_Model_For_Combined_Score: Model identifier for the V2 score
Best_Score: The highest score among all combinations
Best_Source: Which combination produced the best score
v1_v1_Score, v1_v2_Score, v2_v1_Score, v2_v2_Score: Individual scores for each combination
v1_v1_ModelNumber, v1_v2_ModelNumber, v2_v1_ModelNumber, v2_v2_ModelNumber: Model numbers for each score

Plots (`output/PLOTS/` directory)

Scatter plots: Comparing V1 vs V2 reference states
Stacked bar charts: Horizontal and vertical versions with and without special highlighting
Special features: AF3 baseline highlighting, inset zoom regions for specific targets
Full axis plots: Enhanced scatter plots with complete axis ranges
Simple bar plots: Clean, minimal bar chart visualizations
Multi-panel plots: Combined TM-score and GDT_TS score visualizations

Figure S5B. Combined two-state GDT_TS scores for best M1239 models ranked by predictor group. AlphaFold3 Server's Group (304) is highlighted with a star and a horizontal green line.

Figure S7D. Two state GDT_TS scatterplot for T1239. The point corresponding to AlphaFold3 Server's Group (304) is intersected by a vertical and horizontal dashed line.

LaTeX Tables (`output/LATEX_TABLES/` directory)

Publication-ready LaTeX tables for all processed data
Proper formatting and escaping for academic publication
Special T1214 Sigma4 score table

Dual-State LaTeX Tables (`output/DUAL_STATE_LATEX_TABLES/` directory)

LaTeX tables for dual-state score outputs
Publication-ready formatting with proper decimal places
Both regular and longtable formats for large datasets
Proper escaping of special characters for LaTeX

Single-State LaTeX Tables (`output/SINGLE_STATE_LATEX_TABLES/` directory)

LaTeX tables for single-state score outputs
Special T1214 Sigma score tables for Composite_Score_1-4
Publication-ready formatting with proper decimal places
Both regular and longtable formats for large datasets

Manuscript Plots (`output/PLOTS_MANUSCRIPT/` directory)

Publication-ready plots for manuscript figures
Consistent formatting, fonts, and styling
Figure 1C, 2B-C, 4C, 5C, 6B-C, 7C-D and supplementary figures
Both single-state and two-state score visualizations
AF3 baseline highlighting and special formatting

PDF Reports

output/CASP16_TWO_STATE_TABLES_ALL.pdf: Combined PDF of all two-state tables
output/CASP16_TWO_STATE_TABLES.pdf: PDF of two-state tables which appear in main text

Archive (`archive/` directory)

PLOTS/: Archived plot files from previous runs
PDF files: Archived PDF reports from previous processing runs

License

This project is licensed under the MIT License. See the license header in each script file for details.

Author: Tiburon Leon Benavides
Contribution: Main contributor
Date: 2025-09-01

RPIBioinformatics/CASP16_TWO_STATE

About

Resources

Uh oh!

Stars

Watchers

Forks

Releases 1

Packages 0

Uh oh!

Languages

Packages