Skip to main content

🔒 Log in or sign up to get personalized content tailored to your software & hardware background

🔐Sign up or login to translate content

Capstone Project: Embodied Intelligence System

Project Overview​

The capstone project represents the culmination of your learning journey in Physical AI and Humanoid Robotics. You will design, implement, and demonstrate a complete embodied intelligence system that integrates all the concepts covered throughout the course: ROS 2, simulation, NVIDIA Isaac, Vision-Language-Action models, and appropriate hardware platforms.

Project Goals​

The primary objectives of this capstone project are to:

  • Demonstrate comprehensive understanding of embodied intelligence concepts
  • Integrate multiple robotics technologies into a cohesive system
  • Apply learned skills to solve a real-world robotic challenge
  • Develop project management and documentation skills
  • Present technical work to peers and instructors

Project Themes​

Students may choose from the following project themes or propose their own with instructor approval:

Theme 1: Autonomous Mobile Manipulation​

  • Navigate to specified locations
  • Identify and manipulate objects based on visual and language commands
  • Integrate perception, navigation, and manipulation capabilities
  • Demonstrate in simulation and/or on physical hardware

Theme 2: Human-Robot Interaction​

  • Respond to natural language commands for navigation and manipulation
  • Implement social robotics behaviors
  • Demonstrate understanding of context and intent
  • Create engaging and intuitive interaction paradigms

Theme 3: Adaptive Learning Robot​

  • Learn new tasks through demonstration or interaction
  • Adapt behavior based on environmental feedback
  • Implement machine learning techniques for improvement
  • Demonstrate capability transfer across scenarios

Theme 4: Multi-Robot Coordination​

  • Coordinate multiple robots for complex tasks
  • Implement communication and task allocation
  • Demonstrate distributed intelligence
  • Address challenges in multi-agent systems

Project Phases​

Phase 1: Project Proposal and Planning (Week 1-2)​

Deliverables​

  • Project Proposal Document (5 pages max)

    • Problem statement and motivation
    • Technical approach and methodology
    • Integration of course concepts
    • Hardware and software requirements
    • Timeline and milestones
    • Risk assessment and mitigation strategies

  • System Architecture Diagram

    • High-level system design
    • Component interactions
    • Data flow visualization

Evaluation Criteria​

  • Technical feasibility (20%)
  • Integration of course concepts (30%)
  • Clarity of approach (25%)
  • Realism of timeline (25%)

Phase 2: System Implementation (Week 3-8)​

Weekly Milestones​

  • Week 3: Core ROS 2 infrastructure and basic robot setup
  • Week 4: Simulation environment and basic navigation
  • Week 5: Perception system implementation
  • Week 6: NVIDIA Isaac integration and optimization
  • Week 7: VLA model integration and human interaction
  • Week 8: System integration and initial testing

Deliverables​

  • Weekly Progress Reports (1 page each)

    • Accomplishments from previous week
    • Challenges encountered and solutions
    • Plans for upcoming week
    • Updated timeline if necessary

  • Code Repository

    • Well-documented, version-controlled code
    • Clear README with setup instructions
    • Modular design following best practices

Evaluation Criteria​

  • Progress toward milestones (40%)
  • Code quality and documentation (30%)
  • Problem-solving approach (20%)
  • Adaptability to challenges (10%)

Phase 3: Integration and Testing (Week 9-10)​

Activities​

  • Integrate all system components
  • Conduct comprehensive testing
  • Optimize performance
  • Address integration challenges
  • Prepare for demonstration

Deliverables​

  • Integration Report (3 pages)

    • Integration challenges and solutions
    • System performance evaluation
    • Lessons learned
    • Future improvements

  • Testing Documentation

    • Test plans and procedures
    • Results and analysis
    • Performance metrics

Evaluation Criteria​

  • System integration quality (35%)
  • Testing thoroughness (25%)
  • Performance optimization (20%)
  • Problem identification and resolution (20%)

Phase 4: Demonstration and Documentation (Week 11-12)​

Activities​

  • Prepare final demonstration
  • Conduct system testing
  • Create final documentation
  • Prepare presentation materials
  • Peer review and feedback

Deliverables​

  • Final Demonstration

    • Live system demonstration
    • Scenario-based testing
    • Performance validation

  • Final Documentation Package

    • Technical report (10-15 pages)
    • User manual
    • System architecture documentation
    • Code and model repositories

  • Final Presentation (20 minutes)

    • Project overview and motivation
    • Technical approach and implementation
    • Results and evaluation
    • Challenges and lessons learned
    • Future work

Evaluation Criteria​

  • Demonstration quality (30%)
  • Technical achievement (25%)
  • Documentation completeness (20%)
  • Presentation effectiveness (15%)
  • Innovation and creativity (10%)

Technical Requirements​

Mandatory Components​

Every project must include:

  • ROS 2 Integration: All systems must use ROS 2 for communication
  • Perception System: Visual or multi-sensor perception capabilities
  • Navigation/Manipulation: Either mobile navigation or manipulation capabilities
  • NVIDIA Isaac: At least one Isaac component (perception, navigation, or manipulation)
  • VLA Model: Integration of vision-language-action capabilities
  • Simulation: Development and testing in simulation environment
  • Hardware Deployment: Demonstration on appropriate hardware platform

Optional Enhancements​

Projects may include additional features for extra credit:

  • Multi-robot coordination
  • Advanced learning capabilities
  • Novel interaction paradigms
  • Complex task execution
  • Real-world deployment scenarios

Hardware and Software Requirements​

Minimum Hardware​

  • NVIDIA Jetson AGX Orin or equivalent for edge processing
  • RGB-D camera (Intel RealSense, ZED, or equivalent)
  • Robot platform (simulated, TurtleBot3, Unitree Go2, or custom)

Software Stack​

  • ROS 2 Humble Hawksbill
  • Gazebo Garden for simulation
  • NVIDIA Isaac ROS packages
  • Isaac Sim (if using advanced simulation)
  • Python 3.8+ and C++17 for development
  • Git for version control

Cloud Resources (if needed)​

  • AWS/GCP for training large models
  • NVIDIA GPU Cloud for model development
  • Simulation scaling resources

Evaluation Rubric​

Technical Implementation (50%)​

  • System Architecture (15%): Well-designed, modular system
  • Integration Quality (15%): Seamless integration of components
  • Performance (10%): Meets performance requirements
  • Innovation (10%): Novel approaches or solutions

Project Management (20%)​

  • Timeline Adherence (10%): Meeting milestones and deadlines
  • Documentation (10%): Clear, comprehensive documentation

Demonstration and Presentation (20%)​

  • Live Demonstration (10%): Successful system demonstration
  • Presentation Quality (10%): Clear, engaging presentation

Report and Documentation (10%)​

  • Technical Report (5%): Comprehensive technical documentation
  • Code Quality (5%): Well-structured, documented code

Collaboration Guidelines​

Individual Projects​

  • All code and documentation must be individually authored
  • Collaboration limited to general discussions and debugging help
  • Proper attribution required for any shared resources

Team Projects (if approved)​

  • Maximum of 3 students per team
  • Clear division of responsibilities
  • Individual contributions must be documented
  • Peer evaluation of team contributions

Submission Requirements​

Electronic Submission​

  • All deliverables submitted through course management system
  • Code repositories hosted on Git platform
  • Documentation in PDF format
  • Presentation slides in PDF or presentation format

Deadlines​

  • Proposal: End of Week 2
  • Weekly Reports: Every Friday during implementation phase
  • Integration Report: End of Week 10
  • Final Documentation: End of Week 11
  • Final Presentation: During Week 12

Support and Resources​

Instructor Support​

  • Weekly office hours (2 hours)
  • Mid-project review sessions
  • Technical consultation appointments

TA Support​

  • Technical troubleshooting
  • Code review sessions
  • Integration assistance

Peer Support​

  • Project showcase sessions
  • Code review partnerships
  • Collaborative debugging sessions

Assessment Criteria​

Exceeds Expectations (A: 90-100%)​

  • Exceptional technical implementation
  • Innovative solutions to challenges
  • Comprehensive system integration
  • Outstanding presentation and documentation

Meets Expectations (B: 80-89%)​

  • Solid technical implementation
  • Good integration of course concepts
  • Adequate performance and documentation
  • Clear presentation

Approaches Expectations (C: 70-79 Percent)​

  • Basic functionality achieved
  • Some integration challenges
  • Adequate documentation
  • Acceptable presentation

Below Expectations (D/F: Less Than 70 Percent)​

  • Incomplete implementation
  • Significant technical issues
  • Poor documentation
  • Unsatisfactory presentation

Academic Integrity​

All work must be original. Students must:

  • Properly cite all sources and references
  • Document any external code or resources used
  • Clearly indicate collaborative work
  • Follow university academic integrity policies

Accommodations​

Students requiring special accommodations should contact the instructor as soon as possible to discuss appropriate arrangements for the capstone project.

Try With AI​

Try asking your AI companion to help you brainstorm capstone project ideas that combine multiple course concepts, or ask for advice on structuring your project proposal. You can also inquire about common challenges in system integration or ask for guidance on creating effective technical documentation.