Back to work

Mobile + ML inference

StudyAI

AI-native iOS study companion for live lecture capture

Built an iOS study companion that turns in-class live notes and voice capture into structured summaries and machine-generated quizzes, powered by Meta's BART seq2seq model on a FastAPI inference layer with offline-first Firestore and Core Data sync.

Role
Solo Developer
Timeframe
May 2025
Status
Not shipped (iOS prototype)

Product screenshots

StudyAI app screenshot 1 of 6
StudyAI app screenshot 2 of 6
StudyAI app screenshot 3 of 6
StudyAI app screenshot 4 of 6
StudyAI app screenshot 5 of 6
StudyAI app screenshot 6 of 6

Context and problem

Students struggle to convert raw lecture capture—live notes, voice memos, and unstructured material—into review-ready summaries and formative quizzes fast enough to reinforce learning while content is still fresh.

Challenge

Close the gap between live lecture delivery and effective self-review by converting unstructured in-class input into retention-ready summaries and assessments on a mobile-first, latency-sensitive path.

Constraints

  • Mobile-first UX with intermittent campus connectivity—offline access required for post-lecture review
  • Inference latency must stay low enough for interactive study sessions during and after class
  • Quiz quality depends on summarization fidelity from noisy, real-time speech and note input

Key contributions

  • Built in-class live note capture with speech-to-text ingestion, streaming lecture content into the processing pipeline without disrupting session flow.
  • Deployed Meta BART (`facebook/bart-large-cnn`) for abstractive summarization and quiz-item generation—producing study briefs and formative assessment prompts grounded in captured material.
  • Architected an offline-first SwiftUI client with Firestore + Core Data dual persistence and a FastAPI layer targeting p95 inference latency under 400ms.

Architecture

Key engineering decisions

  • Selected Meta BART (`facebook/bart-large-cnn`) for bounded seq2seq summarization and text generation instead of open-ended LLM calls—better cost and latency control for structured study outputs
  • Kept live capture lightweight on-device while delegating summarization and quiz synthesis to the backend inference path
  • Used dual-layer persistence (Firestore + Core Data) so students can review generated material without continuous network access

Security and reliability

  • Offline-first sync for note and quiz access across intermittent campus connectivity
  • Optimized inference paths with p95 response times under 400ms for interactive study workflows

Outcome

Built an iOS study companion prototype that turns in-class live notes and voice capture into BART-powered summaries and auto-generated quizzes—structured review material from raw lecture input in a single mobile workflow.

Limitations

  • Not shipped; iOS prototype only—no App Store release
  • Quiz quality varies with speech recognition accuracy and lecture audio conditions
  • Model scope limited to summarization and generation—no multimodal or long-document RAG pipeline

Next steps

  • Add retrieval grounding for course materials (syllabi, slides) to improve quiz relevance
  • Expand evaluation harnesses for summary and quiz quality across subject domains

Tech stack

  • SwiftUI
  • FastAPI
  • BART (facebook/bart-large-cnn)
  • Firebase
  • Firestore
  • Core Data
  • Speech-to-text