diff --git a/StudentNotebooks/Assignment07_DraftFinalProjectNotebook/lint5-Assignment07-f24.Rmd b/StudentNotebooks/Assignment07_DraftFinalProjectNotebook/lint5-Assignment07-f24.Rmd new file mode 100644 index 0000000..13df53c --- /dev/null +++ b/StudentNotebooks/Assignment07_DraftFinalProjectNotebook/lint5-Assignment07-f24.Rmd @@ -0,0 +1,195 @@ +--- +title: "CTBenchEval App Final Project Report Draft" +author: "Tianyan" +date: "`r Sys.Date()`" +output: + pdf_document: + toc: yes + toc_depth: '3' + html_notebook: default + html_document: + toc: yes + toc_depth: 3 + toc_float: yes + number_sections: yes + theme: united +--- + +# DAR Project and Group Members + +* Project name: CTBenchEval +* Project team members: Xiheng Liu, Yashas Balaji, Ziyi Bao, Corey Curran, Mingyang Li, Corey Curran, Soumeek Mishra, Samuel Park, Victor Zhao + + +# 0.0 Preliminaries. + +This report is generated from an R Markdown file that includes all the R code necessary to produce the results described and embedded in the report. + +```{R, echo=F} +# Set to TRUE to expand R code blocks; set to FALSE to collapse R code blocks +# Code blocks can be surpressed from output for readability using the command code `{R, echo=show}` in the code block header. +show <- FALSE +``` + +Executing this R notebook requires some subset of the following packages: + +* `ggplot2` +* `devtools` +* `conflicted` +* `rmarkdown` +* `knitr` +* `tidyverse` +* `stringr` +* `ggbiplot` +* `pheatmap` +* `plotrix` +* `kableExtra` +* `coin` +* `png` +* `grid` + +These will be installed and loaded as necessary (code suppressed). + + +```{r setup, include=FALSE} + +# Required R package installation; RUN THIS BLOCK BEFORE ATTEMPTING TO KNIT THIS NOTEBOOK!!! +# This section install packages if they are not already installed. +# This block will not be shown in the knit file. +knitr::opts_chunk$set(echo = TRUE) + +# Set the default CRAN repository +local({r <- getOption("repos") + r["CRAN"] <- "http://cran.r-project.org" + options(repos=r) +}) + +if (!require("devtools")) { + install.packages("devtools") + library(devtools) +} +# For package conflict resolution (esp. dplyr functions) +# run con +if (!require("conflicted")) { + devtools::install_github("r-lib/conflicted") + library(conflicted) +} + +# Required packages for CTEval analysis +if (!require("rmarkdown")) { + install.packages("rmarkdown") + library(rmarkdown) +} + +if (!require("knitr")) { + install.packages("knitr") + library(knitr) +} +if (!require("tidyverse")) { + install.packages("tidyverse") + library(tidyverse) +} +# Our preferences +conflicts_prefer(dplyr::summarize()) +conflicts_prefer(dplyr::filter()) +conflicts_prefer(dplyr::select()) +conflicts_prefer(dplyr::mutate()) +conflicts_prefer(dplyr::arrange()) + +if (!require("stringr")) { + install.packages("stringr") + library(stringr) +} + +if (!require("ggbiplot")) { + install.packages("ggbiplot") + library(ggbiplot) +} + +if (!require("pheatmap")) { + install.packages("pheatmap") + library(pheatmap) +} +if (!require("plotrix")) { + install.packages("plotrix") + library(plotrix) +} +if (!require("kableExtra")) { + install.packages("kableExtra") + library(kableExtra) +} + +if (!require("coin")) { + install.packages("coin") + library(coin) +} + +if (!require("png")) { + install.packages("png") + library(png) +} + +if (!require("grid")) { + install.packages("grid") + library(grid) +} +``` + +# 1.0 Project Introduction + +CTBench is a benchmark to assess large language models (LLMs) in aiding clinical study design. + +The CTBenchEval project is to further evaluate and improve the CTBench benchmark. CTBenchEval App is one of the approach. + +# 2.0 Organization of Report + +This report is organize as follows: + +* Section 3.0: User guide of CTBenchEval App + +* Section 4.0 Page 1: 'Step 1: Specify Trial' + +* Section 5.0 Page 2: 'Step 2: Generate Descriptors' + +* Section 6.0 Page 3: 'Step 3: Evaluate' + +# 3.0 User guide of CTBenchEval App + +Begin by pressing "Continue to the app" to assess main webpage. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_01.png") +grid.raster(img) +grid.text("LLM output", x = 0.5, y = 0.95, gp = gpar(fontsize = 20)) +``` + +## 3.1 Data, Code, and Resources + +## 3.2 Contribution + +## 3.3 Methods Description + +## 3.4 Result and Discussion + +## 3.5 Conclusions, Limitations, and Future Work. + +# 4.0 Page 1: 'Step 1: Specify Trial' + +# 5.0 Page 2: 'Step 2: Generate Descriptors' + +# 6.0 Page 3: 'Step 3: Evaluate' + + + +```{R, echo=F} +# Code to read in data if appropriate. + +``` + + +# Bibliography +Provide a listing of references and other sources. + +# Appendix + diff --git a/StudentNotebooks/Assignment07_DraftFinalProjectNotebook/lint5-Assignment07-f24.pdf b/StudentNotebooks/Assignment07_DraftFinalProjectNotebook/lint5-Assignment07-f24.pdf new file mode 100644 index 0000000..f907975 Binary files /dev/null and b/StudentNotebooks/Assignment07_DraftFinalProjectNotebook/lint5-Assignment07-f24.pdf differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_01.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_01.png new file mode 100644 index 0000000..b4bdf83 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_01.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_02.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_02.png new file mode 100644 index 0000000..468a503 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_02.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_03.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_03.png new file mode 100644 index 0000000..61f0df9 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_03.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_04.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_04.png new file mode 100644 index 0000000..1abdbfd Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_04.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_05.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_05.png new file mode 100644 index 0000000..98aedd0 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_05.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_06.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_06.png new file mode 100644 index 0000000..1af0d58 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_06.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_07.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_07.png new file mode 100644 index 0000000..b947dca Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_07.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_08.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_08.png new file mode 100644 index 0000000..1b11475 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_08.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_09.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_09.png new file mode 100644 index 0000000..0592528 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_09.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_10.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_10.png new file mode 100644 index 0000000..dfc1e5f Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_10.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_11.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_11.png new file mode 100644 index 0000000..fac35f9 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_11.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_12.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_12.png new file mode 100644 index 0000000..54d5ae1 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_12.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_13.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_13.png new file mode 100644 index 0000000..f5fa216 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_13.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_14.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_14.png new file mode 100644 index 0000000..ec769d8 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_14.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_15.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_15.png new file mode 100644 index 0000000..7f7eb4e Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_15.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_16.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_16.png new file mode 100644 index 0000000..9dbf6e7 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_16.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_17.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_17.png new file mode 100644 index 0000000..becc317 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_17.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_18.png b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_18.png new file mode 100644 index 0000000..9b3abdd Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_18.png differ diff --git a/StudentNotebooks/FinalProjectNotebook/lint5-FinalNotebook-f24.Rmd b/StudentNotebooks/FinalProjectNotebook/lint5-FinalNotebook-f24.Rmd new file mode 100644 index 0000000..0f937a3 --- /dev/null +++ b/StudentNotebooks/FinalProjectNotebook/lint5-FinalNotebook-f24.Rmd @@ -0,0 +1,346 @@ +--- +title: "CTBenchEval App Final Project Report Draft" +author: "Tianyan" +date: "`r Sys.Date()`" +output: + pdf_document: + toc: yes + toc_depth: '3' + html_notebook: default + html_document: + toc: yes + toc_depth: 3 + toc_float: yes + number_sections: yes + theme: united +--- + +# DAR Project and Group Members + +* Project name: CTBenchEval +* Project team members: Xiheng Liu, Yashas Balaji, Ziyi Bao, Corey Curran, Mingyang Li, Corey Curran, Soumeek Mishra, Samuel Park, Victor Zhao + + +# 0.0 Preliminaries. + +This report is generated from an R Markdown file that includes all the R code necessary to produce the results described and embedded in the report. + +```{R, echo=F} +# Set to TRUE to expand R code blocks; set to FALSE to collapse R code blocks +# Code blocks can be surpressed from output for readability using the command code `{R, echo=show}` in the code block header. +show <- FALSE +``` + +Executing this R notebook requires some subset of the following packages: + +* `ggplot2` +* `devtools` +* `conflicted` +* `rmarkdown` +* `knitr` +* `tidyverse` +* `stringr` +* `ggbiplot` +* `pheatmap` +* `plotrix` +* `kableExtra` +* `coin` +* `png` +* `grid` + +These will be installed and loaded as necessary (code suppressed). + + +```{r setup, include=FALSE} + +# Required R package installation; RUN THIS BLOCK BEFORE ATTEMPTING TO KNIT THIS NOTEBOOK!!! +# This section install packages if they are not already installed. +# This block will not be shown in the knit file. +knitr::opts_chunk$set(echo = TRUE) + +# Set the default CRAN repository +local({r <- getOption("repos") + r["CRAN"] <- "http://cran.r-project.org" + options(repos=r) +}) + +if (!require("devtools")) { + install.packages("devtools") + library(devtools) +} +# For package conflict resolution (esp. dplyr functions) +# run con +if (!require("conflicted")) { + devtools::install_github("r-lib/conflicted") + library(conflicted) +} + +# Required packages for CTEval analysis +if (!require("rmarkdown")) { + install.packages("rmarkdown") + library(rmarkdown) +} + +if (!require("knitr")) { + install.packages("knitr") + library(knitr) +} +if (!require("tidyverse")) { + install.packages("tidyverse") + library(tidyverse) +} +# Our preferences +conflicts_prefer(dplyr::summarize()) +conflicts_prefer(dplyr::filter()) +conflicts_prefer(dplyr::select()) +conflicts_prefer(dplyr::mutate()) +conflicts_prefer(dplyr::arrange()) + +if (!require("stringr")) { + install.packages("stringr") + library(stringr) +} + +if (!require("ggbiplot")) { + install.packages("ggbiplot") + library(ggbiplot) +} + +if (!require("pheatmap")) { + install.packages("pheatmap") + library(pheatmap) +} +if (!require("plotrix")) { + install.packages("plotrix") + library(plotrix) +} +if (!require("kableExtra")) { + install.packages("kableExtra") + library(kableExtra) +} + +if (!require("coin")) { + install.packages("coin") + library(coin) +} + +if (!require("png")) { + install.packages("png") + library(png) +} + +if (!require("grid")) { + install.packages("grid") + library(grid) +} +``` + +# 1.0 Project Introduction + +CTBench is a benchmark to assess large language models (LLMs) in aiding clinical study design. + +The CTBenchEval project is to further evaluate and improve the CTBench benchmark. CTBenchEval App is one of the approach. + +# 2.0 Organization of Report + +This report is organize as follows: + +* Section 3.0: User guide of CTBenchEval App + +* Section 4.0: "Explain" In-Depth + +# 3.0 User guide of CTBenchEval App + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_01.png") +grid.raster(img) +``` + +Begin by pressing "Continue to the app" to assess main webpage. + +## 3.1 Page 1: 'Step 1: Specify Trial' + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_02.png") +grid.raster(img) +``` + +Select or type in (if you know the ID beforehand) the NCT ID that you would like to use from the searchable drop down menu. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_03.png") +grid.raster(img) +``` + +By clicking "Load", you can load the basic information for this NCT id on the right: "Trial ID", "Brief Summary", "Eligibility Criteria", "Conditions", "Primary Outcomes", "Interventions", "Baseline Features". + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_04.png") +grid.raster(img) +``` + +By clicking the button you can download trial information as JSON file. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_05.png") +grid.raster(img) +``` + +Here is a raw example for the JSON file you download. It contains all the information from the NCT id. + +## 3.2 Page 2: 'Step 2: Generate Descriptors' + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_06.png") +grid.raster(img) +``` + +Click the 'Step 2: Generate Descriptors' to switch to next step. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_07.png") +grid.raster(img) +``` + +Upon here you should see the LLM Results of explanation, if not, click "Generate" to see LLM chosen baseline features and a short explain of each feature. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_08.png") +grid.raster(img) +``` + +Scroll down to the bottom, you can also click the button to download the generated report. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_09.png") +grid.raster(img) +``` + +Here is another example for the downloaded json file here. In the report, it contains NCT id, system prompt, user prompt and report. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_10.png") +grid.raster(img) +``` + +In the Step 2's page, you can see "Generation Options" and "Explanation Options" on the left. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_11.png") +grid.raster(img) +``` + +Let us stick with generation options for a while. When you switch to this page, you will see the "Update" button, but we will skip it until all the settings are ready. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_12.png") +grid.raster(img) +``` + +One following setting is that you can choose different model. Since most of our prompts are tested in Llama3, Llama3 is recommended. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_13.png") +grid.raster(img) +``` + +The next thing is the three-shot button. By checking the box, three-shot setting is enabled, you can also choose the three shots that you want to use. You can also modify which three shots you want to choose. The searchable drop down menu is the same as the one from page 1. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_14.png") +grid.raster(img) +``` + +One last thing inside the generation options is the "System Prompt" box and "System Prompt (Three-shot)" box. You can modify LLM's system prompt in either zero shot or three shot here. After you done any settings, you can back to the top of the page to click "Update" to get modified generate result. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_15.png") +grid.raster(img) +``` + +Let's go to Explanation options to finish up the step 2's page. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_16.png") +grid.raster(img) +``` + +The default for explanation is check as default. You can unchecked the box and hit explain to get the raw LLM-chosen baseline feature. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_17.png") +grid.raster(img) +``` + +Open the explain option, there are two major choice of explain. After you chosen the explain choice, hit explain, go back to report you will see it. I will talk more about explain part in the next part. + +# 4.0 "Explain" In-Depth + +## 4.1 Motivation + +During the time making the app, we decide to try expand the app's potential user. A simple report might be a good way to reach this goal. In order to show good visualization for the user, we ask LLM for additional information: For each baseline feature, we ask it for additional explanation. + +## 4.2 Evolution of "Explain" + +The explain prompt start with the simplest: System prompt + "Then for each feature, provide why it is added to this specific trial." If we directly use this prompt, it will have a basic result, but inside the page, it is still plain text. Then I asked it for "bold text" for visible headings; Line break so that it looks more like a report. I modified shiny app input to read all bold text, line breaks and change into and
in the app. The next step is to enhance the output, then the explain options is introduced. As I mentioned upon, if the explain part is not added, the report that users see might only be the definition of the baseline features that LLM chooses. + +One of the best prompt tested without the explain is shown here. + +```{R, echo=F} +# Code +img <- readPNG("../../StudentNotebooks/FinalProjectNotebook/CTBS_Userguide/lint5_CTBS_18.png") +grid.raster(img) +``` + +The full explain prompt goes like this: System Prompt + "Then for each feature, provide why it is added to this specific trial." + Explain + "Use bold text for headings and separate each section with a line break for the explanation." All explain options, result reports are based on this. In between the process of testing prompts, Dr.Bennett has introduced up-to-date concept from clinicaltrials.gov: For the baseline features, age, sex, race and ethnicity, these features are forced in the professional report. For here, one of the prompts that I test is: "If features contain baseline features related to age, gender, ethnicity(NIH/OMB), race, these baseline features should appear first." + +One of the best prompt tested with the explain is shown here. The text is too much for a screenshot, I put plain text here. + +Demographic and Health-related Characteristics:\ +Age 50:\ +To identify the study population's age group and ensure that the participants are elderly overweight individuals.\ +Gender:\ +To determine the gender distribution within the study population and analyze any potential differences in outcomes between males and females.\ +Ethnicity (NIH/OMB):\ +To capture the diverse backgrounds of the participants and assess any potential ethnic differences in outcomes.\ +Race:\ +To identify racial variations within the study population and analyze any potential racial differences in outcomes.\ +Body Mass Index (BMI):\ +To establish the participants' weight status and assess the effectiveness of the weight control program.\ + +Osteoarthritis-related Characteristics:\ +Osteoarthritis Severity:\ +To quantify the severity of osteoarthritis among participants and assess the impact of the exercise and weight control program on disease severity.\ +Number of Joints Affected:\ +To determine the prevalence of osteoarthritis in multiple joints and evaluate the effect of the program on joint-specific outcomes.\ +Behavioral and Health History:\ + +## 4.3 Future work on "explain" + +I have made this successful output as an example for the LLM, but using it as an example will actually lower the chance of producing an actual result. For the current final CTBS app, I have removed the example in order to have a higher chance to produce an example. Future work should be done on this report to make a nicer output for the user. + +# Bibliography + +Link to CTBS Github: https://github.rpi.edu/DataINCITE/DAR-CTBSApp-F24 \ +Link to CTBS App: https://lp01.idea.rpi.edu/shiny/erickj4/DAR-CTBSApp-F24/ \ +Clinical Trial Baseline Descriptor Suggestion Guide: https://docs.google.com/document/d/1rJ_us2p-8T2VToPyVOl6JbfLl5gZ5UpWnOsqvQNEypM/edit?tab=t.0#heading=h.3at9u9s4e0vp \ +Results Data Element Definitions for Interventional and Observational Studies: https://clinicaltrials.gov/policy/results-definitions \ + + diff --git a/StudentNotebooks/FinalProjectNotebook/lint5-FinalNotebook-f24.pdf b/StudentNotebooks/FinalProjectNotebook/lint5-FinalNotebook-f24.pdf new file mode 100644 index 0000000..309bf50 Binary files /dev/null and b/StudentNotebooks/FinalProjectNotebook/lint5-FinalNotebook-f24.pdf differ