RoboActions

1. Initialize RoboActions

Connect to the RoboActions SDK using your API key and project ID.

import roboactions

sdk = roboactions.init(
  api_key="YOUR_API_KEY",
  project_id="my_robot_project"
)

2. Easy Data Collection

Define your sensor configuration and start collecting data seamlessly within your robot's control loop.

# Initialize a Dataset object.
dataset = sdk.new_dataset(
  name="my_robot_data",
  sensor_config={
    "camera": {"resolution": (640, 480), "fps": 30},
    "lidar": {"enabled": True},
    # Add more sensors here
  }
)

# Collect data in your robot loop.
for step in range(100):
  observation = robot.get_observation() # Get sensor data
  action = robot.get_last_action() # Get last action taken
  # Record each observation-action pair
  dataset.record(observation, action)

Save and upload your collected dataset to RoboActions Cloud.

# Save and optionally upload your dataset to RoboActions Cloud
dataset_id = dataset.save(upload=True)
print("Dataset ID:", dataset_id)

3. Scale Up with MimicGen

Augment your training data with MimicGen to improve model robustness and performance.

# Augment data with MimicGen
augmented_dataset_id = sdk.mimicgen(
  dataset_id=dataset_id,
  num_variants=1000,
  augmentation_options={
    "noise_level": 0.05,
    "domain_shift": True
  }
)
print("Augmented Dataset ID:", augmented_dataset_id)

4. Fine-tune and Deploy

Fine-tune a pre-trained Vision-Language Action model (VLA) on your collected and augmented data, then deploy it for inference.

# Fine-tune a base model (e.g., OpenVLA)
model = sdk.finetune(
  base_model="OpenVLA",
  data=dataset_id,
  augmented_data=augmented_dataset_id,
  epochs=5,
  batch_size=16,
  gpu="nvidia-a100",      # or 'auto'
  deploy_for_inference=True
)
print("Fine-tuned Model Accuracy:", model.accuracy)
print("Model ID:", model.model_id)

5. Hybrid Inference

Run inference on your robot or in the cloud, leveraging the best of both worlds for optimal efficiency.

# Run hybrid inference (on-robot if available, otherwise cloud)
prediction = sdk.predict(
  model_id=model.model_id,
  observation=robot.get_observation(),
  mode="hybrid"
)
print("Prediction:", prediction)