From raw trial data to reviewed TLFs

Quick start

HopeAI Data Pipeline takes raw clinical data to SDTM, then ADaM, then tables, listings and figures (TLFs). An AI agent drafts each stage. You check it, change what you disagree with, and approve it.

1. Open Projects and click New project, or open the CDISCPILOT01 demo, which already has sample data.
2. Add your files. The pipeline works out where to start from them. Add anything it lists as missing, then click Create project.
3. Check what the agent read from your raw data in Review data, then confirm.
4. Review each spec in the workspace. Settle the open items, then Approve.
5. The next stage starts by itself. At the end, open Files and download the package.

Projects and the demo

The Project list shows every project with where it started, its stages and its status. Click a row to open the project at the step it is on.

The row tagged Demo is CDISCPILOT01 with sample files included: click it to create the project and walk through the whole pipeline. Rows tagged Sample only show what a busy list looks like and do not open.

Use New project at the top right of the list to start your own. Every project you create stays in the list with its own files, progress and chat, so you can leave one and come back to it later. A build that is running waits while you are in another project and carries on when you return.

Create a project

Name the project, then add your files: choose them, drag them in, or use Add CDISCPILOT01 sample files. Each file gets a type automatically (raw data, aCRF, SDTM datasets, SAP, map spec, shells...). Change a type from the list if it is wrong, or remove a file with the ×.

• Where it starts. Raw data or an aCRF means SDTM comes first. SDTM datasets alone mean it starts at ADaM. ADaM datasets alone mean it starts at TLF. You can change Start at, and pick Finish at to stop early, for example after SDTM.
• Who writes the spec. Under Map spec to use (or Shells to use) choose Draft with AI, I will upload my own, or one of your templates.
• Required files. If something the pipeline needs is missing, for example the SAP, it is listed under Required files missing and Create project stays grey until you add it.
• Incomplete CRFs. If the aCRF does not cover every form in the raw data, you are told which forms are missing. You can add them, or go on: those domains are then drafted from the raw data columns only and their variables are marked no aCRF for your review.
• Standards and language. Phase, the SDTM and ADaM IG versions and the output language (SAS or R) sit in a folded section. The IG is the CDISC rulebook for SDTM and ADaM. Keep the latest unless your sponsor or the agency asked for an older version for this submission.

Creating the project takes you straight to the first step. You can change the project name, phase and IG versions later by clicking the project name in the top bar.

Review data

If you start at SDTM, the first page checks how your raw file was read. The left card shows the original file, sheet by sheet, with the problem records highlighted. The Checks list on the right has what the agent found: duplicate and deleted records, unit conflicts, missing information, date formats and column mapping.

• Click a check to see its rows in the file.
• Each check has options with the recommended one marked. Pick one, or tell the agent in the chat in plain words, for example these are cm, use the range rule. A change is shown first with Apply and Cancel.
• When you are happy, click Confirm input and generate SDTM draft at the bottom.

The workspace

The workspace is one card on the left and the Copilot chat on the right.

• At the top of the card, a line shows the project: three big circles for SDTM, ADaM and TLF, and small circles for the steps inside each one (Check, Map spec, Datasets; Shells, Tables, Delivery). Green is done, purple is where you are.
• Inputs (left) shows your source: the aCRF or raw sheets, the SAP, or your shells. Click something in the middle panel and the matching place is highlighted here.
• Derived outputs (middle) holds the variable list on the Map spec tab and the built Datasets or TLFs after approval.
• Drag the thin bars to resize: between Inputs and Derived outputs, and between the card and the chat. Double-click a bar to reset it. The chat never gets narrower than a readable width.
• The Copilot button in the top bar hides or shows the chat.

Watching the agent work

While the agent works, the chat shows what it is doing right now: which file it is reading, what it is matching, which dataset it is building. Steps tick off one by one. When it finishes, the log folds into one line, for example Reading your files · 6 steps · 4 s, which you can open again, and the result appears below it.

Use Pause in the bar above the chat input to hold the agent, and Resume to let it carry on.

Review the spec

Every variable has a status: Needs you, accepted (a tick), Edited, or locked after approval.

• On the right of each row there are buttons. A row that needs you has Decide, which opens its question in the chat, and Accept, which takes the recommended answer. Other rows have Change.
• Click a row to see its rule, where it comes from (Source) and its version. The chat input is filled with VAR: change to . Finish the sentence and send. Ticked rows were accepted automatically and can be changed too.
• The card in the chat can be closed with the ×. You can always come back through the row buttons or Review N open items in the sign-off bar.
• You can also ask a question instead of changing, for example why is it derived this way?.

Talk to the agent

The chat understands plain requests. Under the input there are Try suggestions for the page you are on. Some examples:

• Show the flagged variables, next issue, previous issue
• AESTDY: change to study day relative to RFSTDTC
• Where does AESEV come from?
• What variables are in DS?, open ADSL
• Show table 14-1.02, Which tables differ from the reference?
• Accept all clean rows, What is the status?, help

Press Tab on an empty input to fill the example shown in it. You can attach files with the paperclip.

Approve and move on

When nothing is open, Approve in the sign-off bar locks the spec as v1.0 and builds the datasets. The chat shows the build step by step, then the result.

The next stage then starts by itself after a short countdown, if its files are in place. If something is missing, the agent asks for it in the chat: attach it with the paperclip or use the sample files. You can Pause at any time.

Before the TLF stage the agent asks whether you have your own mock shells. If not, it drafts them from the SAP.

The sign-off bar also has Add to my templates, which saves the current spec or shells for reuse.

Tables and shells

In the TLF stage the middle panel lists every output on the left and shows the selected one on the right. Switch between the shell and the version with numbers, and use Enlarge to open it full size. A badge tells you whether the numbers match the reference run and which SAP page the table follows.

Approve the shells to fill the tables. Outputs that this demo cannot fill stay as shells.

Trace any number

Results can be traced back at three levels:

• Column. Click a variable to see its rule and the source column.
• Cell. Click a value in a dataset or a table to see how it was calculated.
• Record. Click a row to jump to the exact source record in the raw sheet.

If you change a rule later, earlier outputs keep the version they were built from, so each dataset stays tied to the spec that made it.

Files and delivery

The Files page lists everything in groups: Uploaded by you, then SDTM, ADaM, TLF and other generated files. Open or close a group by its header, or use Expand all and Collapse all. Inside a group the files are sorted by type: datasets, map spec, programs, tables and figures, reports.

Click a row to preview it, or its download arrow to save that file. Download package gives you everything in one zip. When a run finishes, See all files takes you here.

My templates

Templates belong to your account, not to a project, so you can reuse them everywhere.

• In the workspace, click Add to my templates in the sign-off bar to save the current SDTM spec, ADaM spec or shells.
• Open your name menu (top right) and choose My templates to download or remove them, or to upload your own.
• When you create a project, pick a template under Map spec to use or Shells to use. Its rules are imported into the spec: the chat tells you how many replaced the draft, how many already matched and how many do not exist in this study.

The same menu has User guide, Theme and Sign out.

FAQ

快速开始

HopeAI Data Pipeline 把原始临床数据依次做成 SDTM、ADaM，再做成统计表、列表和图（TLF）。每个阶段由 AI agent 先起草，你来检查、修改不同意的地方，然后批准。

1. 打开 Projects，点 New project，或者直接打开已带示例数据的 CDISCPILOT01 演示项目。
2. 添加文件。系统会根据文件判断从哪一步开始；补齐它提示缺少的文件，再点 Create project。
3. 在 Review data 里检查 agent 对原始数据的读取结果，然后确认。
4. 在工作台里审阅每个阶段的 spec，处理完待办项后点 Approve。
5. 下一个阶段会自动开始。全部完成后打开 Files 下载交付包。

项目列表与演示项目

Project list 列出所有项目的起点、阶段和状态。点一行就会回到该项目当前所在的步骤。

带 Demo 标签的是 CDISCPILOT01，自带示例文件：点它就会创建项目，你可以走完整条流程。带 Sample 标签的行只是展示列表的样子，点了不会打开。

要开始自己的项目，点列表右上角的 New project。你创建的每个项目都会留在列表里，各自保留文件、进度和聊天记录，离开后可以随时回来继续。正在运行的构建在你切到别的项目时会等待，回来后继续。

新建项目

先填项目名，再添加文件：可以选择文件、直接拖进来，或者点 Add CDISCPILOT01 sample files。每个文件会自动判断类型（原始数据、aCRF、SDTM 数据集、SAP、map spec、shells 等）。类型不对可以在列表里改，用 × 可以移除文件。

• 从哪里开始。有原始数据或 aCRF，就从 SDTM 开始；只有 SDTM 数据集，就从 ADaM 开始；只有 ADaM 数据集，就从 TLF 开始。你可以改 Start at，也可以用 Finish at 提前结束，比如做完 SDTM 就停。
• spec 谁来写。在 Map spec to use（TLF 是 Shells to use）里选 Draft with AI、I will upload my own（自己上传），或者选一个我的模板。
• 必需文件。如果缺少流程需要的文件（比如 SAP），会列在 Required files missing 下面，补齐之前 Create project 保持灰色。
• CRF 不全。如果 aCRF 没有覆盖原始数据里的所有表单，会告诉你缺哪几个。你可以补上，也可以继续：这些域只能根据原始数据的列名起草，相关变量会标上 no aCRF 供你重点检查。
• 标准与语言。阶段（Phase）、SDTM 和 ADaM 的 IG 版本、输出语言（SAS 或 R）收在折叠区里。IG 是 CDISC 为 SDTM 和 ADaM 制定的规则手册。除非申办方或监管机构要求本次提交用旧版本，否则保持最新即可。

创建后会直接进入第一步。之后想改项目名、阶段或 IG 版本，点顶部栏里的项目名即可。

Review data（检查数据读取）

如果从 SDTM 开始，第一页检查原始文件是否读对了。左边卡片按工作表显示原始文件，问题记录会高亮；右边的 Checks 列出 agent 发现的问题：重复与已删除记录、单位冲突、缺失信息、日期格式、列映射。

• 点一个检查，就能在文件里看到对应的行。
• 每个检查都有几个选项，推荐项有标记。可以直接选，也可以在聊天里用大白话告诉 agent，比如 these are cm, use the range rule。修改会先给出预览，带 Apply 和 Cancel。
• 满意之后，点底部的 Confirm input and generate SDTM draft。

工作台

工作台左边是一张卡片，右边是 Copilot 聊天栏。

• 卡片最上面是项目进度线：三个大圈代表 SDTM、ADaM、TLF，每个大圈后面的小圈是它的步骤（Check、Map spec、Datasets；Shells、Tables、Delivery）。绿色表示已完成，紫色表示当前所在。
• Inputs（左）是你的来源材料：aCRF 或原始工作表、SAP、你的 shells。点中间面板里的内容，这里对应位置会高亮。
• Derived outputs（中）在 Map spec 页签里是变量列表，批准后会出现生成的 Datasets 或 TLFs。
• 拖动细条可以调整宽度：Inputs 和 Derived outputs 之间，以及卡片和聊天栏之间。双击拖拽条恢复默认。聊天栏不会窄到看不清。
• 顶部栏的 Copilot 按钮可以收起或展开聊天栏。

看 agent 工作进度

agent 工作时，聊天栏会实时显示它正在做什么：读哪个文件、在匹配什么、在生成哪个数据集，步骤一步步打勾。完成后，这段记录会折叠成一行，例如 Reading your files · 6 steps · 4 s，点开可以再看，结果显示在它下面。

聊天输入框上方的栏里有 Pause，可以让 agent 暂停；点 Resume 继续。

审阅 spec

每个变量都有状态：Needs you（需要你处理）、已接受（打勾）、Edited（已改）、批准后锁定。

• 每一行右边有按钮。需要你处理的行有 Decide（在聊天里打开对应问题）和 Accept（采用推荐答案）；其他行有 Change。
• 点一行可以看到它的 规则、来源（Source）和版本，聊天输入框会自动填入 VAR: change to ，补完这句话发送即可。打勾的行是自动接受的，也可以改。
• 聊天里的卡片可以用 × 关掉，之后随时可以通过行上的按钮，或签字栏里的 Review N open items 重新打开。
• 也可以不改，只提问，比如 why is it derived this way?。

和 agent 对话

聊天可以理解日常说法。输入框上方有针对当前页面的 Try 建议。一些例子：

• Show the flagged variables、next issue、previous issue
• AESTDY: change to study day relative to RFSTDTC
• Where does AESEV come from?
• What variables are in DS?、open ADSL
• Show table 14-1.02、Which tables differ from the reference?
• Accept all clean rows、What is the status?、help

输入框为空时按 Tab 可以填入框里显示的示例。用回形针可以附加文件。

批准并进入下一步

没有待处理项后，点签字栏里的 Approve，spec 会锁定为 v1.0 并开始生成数据集，聊天栏会一步步显示构建过程，然后给出结果。

如果下一阶段需要的文件都齐了，它会在短暂倒计时后自动开始。如果缺文件，agent 会在聊天里问你要：用回形针附加，或者使用示例文件。你可以随时 Pause。

进入 TLF 阶段之前，agent 会问你有没有自己的 mock shells；没有的话，它根据 SAP 起草。

签字栏里还有 Add to my templates，可以把当前 spec 或 shells 存下来以后复用。

表格与 shells

TLF 阶段，中间面板左侧列出所有输出，右侧显示选中的那一个。可以在 shell 和 with numbers（带数字）之间切换，点 Enlarge 放大查看。标签会告诉你数字是否和参考结果一致，以及这张表对应 SAP 的哪一页。

批准 shells 后表格会被填入数字。这个演示里无法生成的输出会保持为 shell。

追溯任何一个数字

结果可以从三个层面追溯：

• 列。点一个变量，查看它的规则和来源列。
• 单元格。点数据集或表格里的一个值，查看它是怎么算出来的。
• 记录。点一行，会跳到原始工作表里对应的那条记录。

之后如果改了规则，之前生成的输出仍保留它们当时所用的版本，所以每个数据集都对应生成它的那一版 spec。

文件与交付

Files 页把所有文件分组列出：先是 Uploaded by you（你上传的），然后是 SDTM、ADaM、TLF 和其他生成的文件。点分组标题可以展开或收起，也可以用 Expand all 和 Collapse all。组内按类型排序：数据集、map spec、程序、表和图、报告。

点一行预览，点它的下载箭头保存这个文件。Download package 一次下载全部文件（zip）。流程跑完后，See all files 会带你来到这里。

我的模板

模板属于你的账号，不属于某个项目，所以可以在所有项目里复用。

• 在工作台里，点签字栏的 Add to my templates，保存当前的 SDTM spec、ADaM spec 或 shells。
• 点右上角的用户头像菜单，选 My templates，可以下载或删除模板，也可以上传你自己的。
• 新建项目时，在 Map spec to use 或 Shells to use 里选模板。模板里的规则会导入 spec：聊天里会告诉你有多少条替换了草稿、多少条本来就一致、多少条在这个研究里不存在。

同一个菜单里还有 User guide、Theme 和 Sign out。

常见问题