Scholaryx Sequential multi-agent research pipeline Connecting…

Agentic AI · tool-using LLM agents · LLM-as-judge

Multi-agent research workbench

Four components run strictly in sequence: a search agent queries the web, a reader agent scrapes the most relevant page, a writer chain drafts a structured report, and a critic chain scores it. Every output, timing and token count below comes from the actual run.

Pipeline

No run yet

idle
  1. 01waiting
    Search agent
    Tool-calling agent → web_search
    —
  2. 02waiting
    Reader agent
    Tool-calling agent → scrape_url
    —
  3. 03waiting
    Writer chain
    Prompt → LLM → report
    —
  4. 04waiting
    Critic chain
    Prompt → LLM → score /10
    —
No report yet Enter a topic and run the pipeline. The writer chain's report appears here.

Architecture

The pipeline is sequential: each component consumes the previous one's text output. There is no planner, no parallel execution and no feedback loop from the critic back to the writer.

  1. InputResearch topic
  2. 01 · Search agent LangChain create_agent with gpt-4o-mini and one tool, web_search (Tavily, 5 results). Returns the agent's final message.
  3. 02 · Reader agent create_agent with scrape_url. Receives the first 800 characters of the search output, picks one URL, returns up to 3,000 characters of cleaned page text.
  4. 03 · Writer chain ChatPromptTemplate | llm | StrOutputParser. Required sections: Introduction, Key Findings (≥ 3), Conclusion, Sources.
  5. 04 · Critic chain Same LLM, separate prompt. Returns Score: X/10, strengths, areas to improve and a one-line verdict.
  6. OutputReport + critique

Backend

server.py (FastAPI) starts each run in a worker thread and calls the existing pipeline.run_research_pipeline(topic) unchanged. Stage progress is read from the step N lines that function prints.

Frontend

Plain HTML, CSS and JavaScript. The page polls GET /api/runs/{id} once per second; reports are rendered with an escape-first Markdown renderer, so model output cannot inject HTML.

Secrets

OPENAI_API_KEY and TAVILY_API_KEY are read from the server environment only. The browser receives just whether each key is configured.

Limitations

  • The critic is the same model grading its own pipeline. The score is an LLM-as-judge opinion with no reference answer; it is not a validated quality metric and can vary between runs.
  • Narrow evidence. The reader sees only the first 800 characters of the search output and scrapes a single page, truncated to 3,000 characters.
  • No citation verification in the pipeline. The writer is asked to list URLs; nothing checks them. The Sources tab adds a UI-side cross-check against the gathered material.
  • Not reproducible bit-for-bit. Temperature is 0, but web search results and pages change over time.
  • No streaming of intermediate text. The pipeline returns all outputs at the end; during the run, only stage progress and the console are live.
  • In-memory server state. Runs on the server are lost on restart; only completed runs saved in your browser persist.