GPU & CPU On-Demand Rental

Spin up dedicated NVIDIA A10G and T4 GPU nodes or scalable CPU instances in seconds via API, SDK, or cURL.

Rentals provide full root SSH access, persistent EBS volumes, customizable security groups, automated software presets for ML inference (vLLM, ComfyUI, PyTorch), serial console logging, and dynamic status telemetry.

Platform Overview

FotoHub Compute operates exclusively in the eu-central-1 (Frankfurt) region. Billing is processed exclusively in USD from a prepaid wallet. Credits or other currencies (e.g., PLN) are strictly not supported. A minimum balance of $0.50 USD is required to provision any instance.

GPU Family Catalog & Hardware Specifications

We currently support 22 total instance types across the T3, C5, M5, R5, G4dn, and G5 families. Below is a deep dive into our flagship GPU instances for ML workloads.

G5 Instances (NVIDIA A10G)

The G5 family is powered by NVIDIA A10G Tensor Core GPUs, offering exceptional performance for graphics-intensive applications and machine learning inference.

VRAM: 24 GB GDDR6 per GPU
CUDA Cores: 9,216
Tensor Cores: 288 (3rd Gen)
FP16 / BF16 Performance: 125 TFLOPS (250 TFLOPS with sparsity)
TF32 Performance: 62.5 TFLOPS (125 TFLOPS with sparsity)
Pricing (G5.xlarge):
- Spot: $0.38/hr
- On-Demand: $1.01/hr

Recommended Use Cases

Ideal for large language models (LLMs) via vLLM, high-resolution Stable Diffusion (SDXL/Flux) generation, and complex deep learning training jobs.

G4dn Instances (NVIDIA T4)

The G4dn family provides cost-effective inference capabilities via NVIDIA T4 GPUs.

VRAM: 16 GB GDDR6 per GPU
CUDA Cores: 2,560
Tensor Cores: 320 (Turing)
FP16 Performance: 65 TFLOPS
INT8 Performance: 130 TOPS
Pricing (G4dn.xlarge):
- Spot: $0.20/hr
- On-Demand: $0.53/hr

Cost Efficiency

Perfect for smaller models (e.g., Llama-3-8B), standard Stable Diffusion 1.5/XL, and continuous background inference tasks where cost is the primary factor.

Spot vs On-Demand Decision

Deciding between Spot and On-Demand instances depends on your workload's fault tolerance and duration. Spot instances offer up to a 70% discount but can be interrupted with minimal notice.

mermaid

flowchart TD
    A[Start: Need compute] --> B{Is workload fault-tolerant?}
    B -- Yes --> C{Duration < 8 hours?}
    B -- No --> D[Choose On-Demand]
    C -- Yes --> E[Choose Spot Instance]
    C -- No --> F{Can you handle interruption?}
    F -- Yes --> E
    F -- No --> D
    E --> G[Save up to ~62-70%]
    D --> H[Guaranteed uptime]

Spot Interruptions

Spot instances may be reclaimed by the platform when overall capacity is low. Always ensure your scripts periodically save checkpoints to your root or attached EBS volume.

Billing & Cost Management

Billing is calculated transparently using an hourly model based entirely on USD.

Key Billing Rules

Currency: 100% USD from your prepaid wallet.
Minimum Requirement: You must have at least $0.50 USD in your wallet to provision any instance.
Deduction: Costs are deducted automatically based on the hourly rate. Partial hours are billed per second after the first minute.
Tracking: You can view real-time charges and active instances in the /console dashboard.

Cost Calculation Formula

Total Cost = (Instance Hourly Rate + (EBS Volume GB * Storage Hourly Rate per GB)) * Runtime Hours

Example calculation for G5.xlarge Spot ($0.38/hr) with 100GB gp3 storage ($0.08/GB/month = ~$0.00011/GB/hr) for 10 hours:

Instance: $0.38 * 10 = $3.80
Storage: 100 * $0.00011 * 10 = $0.11
Total: $3.91

Lifecycle API Summary

Base URL: https://apis.fotohub.app/compute/v1
Authentication: Authorization: Bearer fh_live_YOUR_API_KEY

Action	Endpoint	Description
Catalog	`GET /catalog`	List real-time prices, availability zones, and specs
Eligibility	`GET /instances/eligibility`	Verify account tier & wallet balance sufficiency
Estimate	`POST /instances/estimate`	Calculate hourly and total costs prior to launching
Provision	`POST /instances`	Spin up an on-demand or Spot instance
Instance Detail	`GET /instances/{id}/full`	Inspect network IPs, storage attachments, and status
Power State	`POST /instances/{id}/start	stop
Resize	`POST /instances/{id}/resize`	Hot-swap instance type (e.g. `g4dn.xlarge` → `g5.2xlarge`)
SSH Key	`GET /instances/{id}/ssh-key`	Download injected private SSH `.pem` key
Regenerate SSH	`POST /instances/{id}/ssh-key/regenerate`	Re-inject fresh SSH credentials
Serial Console	`GET /instances/{id}/console-output`	Real-time EC2 serial kernel output & boot log
CloudWatch Metrics	`GET /instances/{id}/metrics`	Query CPU, GPU VRAM, disk IOPS, and network I/O
Status Checks	`GET /instances/{id}/status-checks`	Query AWS System and Instance status checks
Run Script	`POST /instances/{id}/run-script`	Execute remote shell script on running instance

Step-by-Step Guide: Provisioning a GPU Node

A complete lifecycle flow involves checking eligibility, estimating costs, provisioning, waiting for the active state, and connecting via SSH.

1. Check Eligibility

First, ensure you have sufficient funds ($0.50 USD minimum).

cURLPythonTypeScriptGo

bash

curl -X GET https://apis.fotohub.app/compute/v1/instances/eligibility \
  -H "Authorization: Bearer $FOTOHUB_API_KEY"

python

import requests, os
res = requests.get("https://apis.fotohub.app/compute/v1/instances/eligibility", headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"})
print(res.json())

typescript

const res = await fetch("https://apis.fotohub.app/compute/v1/instances/eligibility", {
  headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}` }
});
console.log(await res.json());

req, _ := http.NewRequest("GET", "https://apis.fotohub.app/compute/v1/instances/eligibility", nil)
req.Header.Set("Authorization", "Bearer "+os.Getenv("FOTOHUB_API_KEY"))
client := &http.Client{}
resp, _ := client.Do(req)

2. Estimate Costs

Evaluate the exact price of your configuration.

cURLPythonTypeScriptGo

bash

curl -X POST https://apis.fotohub.app/compute/v1/instances/estimate \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"catalog_id": "g5.xlarge", "spot_instance": true, "root_volume_size_gb": 120, "max_runtime_hours": 8}'

python

import requests, os
payload = {"catalog_id": "g5.xlarge", "spot_instance": True, "root_volume_size_gb": 120, "max_runtime_hours": 8}
res = requests.post("https://apis.fotohub.app/compute/v1/instances/estimate", json=payload, headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"})
print(res.json())

typescript

const payload = { catalog_id: "g5.xlarge", spot_instance: true, root_volume_size_gb: 120, max_runtime_hours: 8 };
const res = await fetch("https://apis.fotohub.app/compute/v1/instances/estimate", {
  method: "POST", headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}`, "Content-Type": "application/json" }, body: JSON.stringify(payload)
});

payload := map[string]interface{}{"catalog_id": "g5.xlarge", "spot_instance": true, "root_volume_size_gb": 120, "max_runtime_hours": 8}
jsonPayload, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://apis.fotohub.app/compute/v1/instances/estimate", bytes.NewBuffer(jsonPayload))
req.Header.Set("Authorization", "Bearer "+os.Getenv("FOTOHUB_API_KEY"))
req.Header.Set("Content-Type", "application/json")

3. Provision the Instance

Launch the instance in eu-central-1.

cURLPythonTypeScriptGo

bash

curl -X POST https://apis.fotohub.app/compute/v1/instances \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "my-gpu-node", "catalog_id": "g5.xlarge", "spot_instance": true,
    "max_runtime_hours": 8, "root_volume_size_gb": 120, "region": "eu-central-1",
    "install_presets": ["docker", "python-ml"]
  }'

python

payload = {
    "name": "my-gpu-node", "catalog_id": "g5.xlarge", "spot_instance": True,
    "max_runtime_hours": 8, "root_volume_size_gb": 120, "region": "eu-central-1",
    "install_presets": ["docker", "python-ml"]
}
res = requests.post("https://apis.fotohub.app/compute/v1/instances", json=payload, headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"})
instance_id = res.json()["instance"]["id"]

typescript

const payload = {
    name: "my-gpu-node", catalog_id: "g5.xlarge", spot_instance: true,
    max_runtime_hours: 8, root_volume_size_gb: 120, region: "eu-central-1",
    install_presets: ["docker", "python-ml"]
};
const res = await fetch("https://apis.fotohub.app/compute/v1/instances", {
  method: "POST", headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}`, "Content-Type": "application/json" }, body: JSON.stringify(payload)
});

payload := map[string]interface{}{"name": "my-gpu-node", "catalog_id": "g5.xlarge", "spot_instance": true, "max_runtime_hours": 8, "root_volume_size_gb": 120, "region": "eu-central-1", "install_presets": []string{"docker", "python-ml"}}
jsonPayload, _ := json.Marshal(payload)
req, _ := http.NewRequest("POST", "https://apis.fotohub.app/compute/v1/instances", bytes.NewBuffer(jsonPayload))
req.Header.Set("Authorization", "Bearer "+os.Getenv("FOTOHUB_API_KEY"))
req.Header.Set("Content-Type", "application/json")

4. Wait for Running & Fetch IP (Polling Loop)

Instances take 1-3 minutes to become ready. Use a polling loop.

PythonTypeScriptGo

python

import time
while True:
    res = requests.get(f"https://apis.fotohub.app/compute/v1/instances/{instance_id}/full", headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"}).json()
    status = res["instance"]["status"]
    ip = res["instance"].get("ip_address")
    if status == "running" and ip:
        print(f"Ready! IP: {ip}")
        break
    print("Waiting...")
    time.sleep(10)

typescript

let isRunning = false;
let ipAddress = "";
while (!isRunning) {
  const res = await fetch(`https://apis.fotohub.app/compute/v1/instances/${instanceId}/full`, {
    headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}` }
  });
  const data = await res.json();
  if (data.instance.status === "running" && data.instance.ip_address) {
    isRunning = true;
    ipAddress = data.instance.ip_address;
    console.log(`Ready! IP: ${ipAddress}`);
  } else {
    console.log("Waiting...");
    await new Promise(r => setTimeout(r, 10000));
  }
}

// Simplified loop
for {
    req, _ := http.NewRequest("GET", fmt.Sprintf("https://apis.fotohub.app/compute/v1/instances/%s/full", instanceID), nil)
    req.Header.Set("Authorization", "Bearer "+os.Getenv("FOTOHUB_API_KEY"))
    resp, _ := http.DefaultClient.Do(req)
    // Decode JSON, check status...
    time.Sleep(10 * time.Second)
}

Download your injected PEM key securely.

PythonTypeScriptcURL

python

key_res = requests.get(f"https://apis.fotohub.app/compute/v1/instances/{instance_id}/ssh-key", headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"}).json()
with open("key.pem", "w") as f:
    f.write(key_res["private_key"])
os.chmod("key.pem", 0o400)
print(f"ssh -i key.pem ubuntu@{ip}")

typescript

import * as fs from "fs";
const keyRes = await fetch(`https://apis.fotohub.app/compute/v1/instances/${instanceId}/ssh-key`, {
  headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}` }
});
const keyData = await keyRes.json();
fs.writeFileSync("key.pem", keyData.private_key, { mode: 0o400 });
console.log(`ssh -i key.pem ubuntu@${ipAddress}`);

bash

curl -X GET https://apis.fotohub.app/compute/v1/instances/inst_123/ssh-key \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" | jq -r .private_key > key.pem
chmod 400 key.pem

Lifecycle Operations Guide

Manage your instances dynamically.

Start, Stop, Reboot, Terminate

Replace {action} with start, stop, reboot, or terminate.

cURLPythonTypeScriptGo

bash

curl -X POST https://apis.fotohub.app/compute/v1/instances/inst_123/stop \
  -H "Authorization: Bearer $FOTOHUB_API_KEY"

python

requests.post(f"https://apis.fotohub.app/compute/v1/instances/{instance_id}/stop", headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"})

typescript

await fetch(`https://apis.fotohub.app/compute/v1/instances/${instanceId}/stop`, { method: "POST", headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}` } });

req, _ := http.NewRequest("POST", fmt.Sprintf("https://apis.fotohub.app/compute/v1/instances/%s/stop", instanceID), nil)
req.Header.Set("Authorization", "Bearer "+os.Getenv("FOTOHUB_API_KEY"))
http.DefaultClient.Do(req)

Dynamic Vertical Resizing (`POST /resize`)

Upgrade or downgrade compute hardware without reconfiguring operating systems, reinstalling CUDA libraries, or re-downloading model weights.

Resize Constraints

The instance must be in a stopped state before resizing. You can only resize to a catalog tier that supports your current root volume architecture.

cURLPythonTypeScript

bash

curl -X POST https://apis.fotohub.app/compute/v1/instances/inst_123/resize \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"instance_type": "g5.2xlarge"}'

python

requests.post(f"https://apis.fotohub.app/compute/v1/instances/{instance_id}/resize", json={"instance_type": "g5.2xlarge"}, headers={"Authorization": f"Bearer {os.getenv('FOTOHUB_API_KEY')}"})

typescript

await fetch(`https://apis.fotohub.app/compute/v1/instances/${instanceId}/resize`, { 
  method: "POST", 
  headers: { "Authorization": `Bearer ${process.env.FOTOHUB_API_KEY}`, "Content-Type": "application/json" },
  body: JSON.stringify({ instance_type: "g5.2xlarge" })
});

Install Presets Reference

The install_presets array allows you to inject bootstrap logic automatically.

Preset Name	Description	Key Software Installed
`docker`	Container execution environment	Docker CE, Docker Compose
`nvidia-cuda`	Core GPU drivers and toolkit	NVIDIA Drivers 535+, CUDA 12.x, cuDNN
`nvidia-docker`	GPU passthrough for containers	NVIDIA Container Toolkit
`python-ml`	Data science base	Python 3.10+, pip, virtualenv, PyTorch, xformers
`huggingface`	Model hub integration	`huggingface_hub`, `transformers`, `accelerate`
`vllm`	High-throughput LLM server	vLLM engine, OpenAI API server
`comfyui`	Node-based Stable Diffusion	ComfyUI, custom nodes manager, essential workflows

Security Group Rules Reference

By default, all ports are blocked except for basic outgoing traffic. Open specific ports using the security_group_rules array.

Application / Purpose	Protocol	Port	Example Rule
SSH	TCP	22	`{"protocol": "tcp", "port": 22, "cidr": "0.0.0.0/0", "description": "SSH Access"}`
ComfyUI	TCP	8188	`{"protocol": "tcp", "port": 8188, "cidr": "0.0.0.0/0", "description": "ComfyUI Interface"}`
vLLM API	TCP	8000	`{"protocol": "tcp", "port": 8000, "cidr": "0.0.0.0/0", "description": "vLLM OpenAI Server"}`
Gradio / SD WebUI	TCP	7860	`{"protocol": "tcp", "port": 7860, "cidr": "0.0.0.0/0", "description": "Gradio App"}`
HTTPS Web	TCP	443	`{"protocol": "tcp", "port": 443, "cidr": "0.0.0.0/0", "description": "Secure Web"}`
FastAPI / Express	TCP	8080	`{"protocol": "tcp", "port": 8080, "cidr": "0.0.0.0/0", "description": "Custom API Server"}`

Cloud-Init & ML Stack Examples

Pass shell scripts via the startup_script parameter to bootstrap your environment automatically upon boot.

PyTorch & HuggingFace Setup

bash

#!/bin/bash
export DEBIAN_FRONTEND=noninteractive
apt-get update && apt-get install -y git python3-pip
pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip3 install transformers huggingface_hub accelerate
# Cache directory configuration
mkdir -p /mnt/models/hf_cache
echo "export HF_HOME=/mnt/models/hf_cache" >> /home/ubuntu/.bashrc

vLLM & DeepSeek Setup

bash

#!/bin/bash
# Assumes docker & nvidia-docker presets are enabled
docker run -d --gpus all -p 8000:8000 --ipc=host --name vllm \
  -v /data/hf_cache:/root/.cache/huggingface \
  vllm/vllm-openai:latest \
  --model deepseek-ai/DeepSeek-R1-Distill-Qwen-8B --port 8000 --max-model-len 4096

5 Real-World Provisioning Scenarios

Scenario 1: Provisioning a ComfyUI Worker (NVIDIA A10G)

Deploy an NVIDIA A10G 24GB node (g5.xlarge) with Docker and NVIDIA Container Toolkit pre-installed, booting ComfyUI directly on port 8188.

Python

python

from fotohub import FotoHub
import os, time

client = FotoHub(api_key=os.environ["FOTOHUB_API_KEY"])

instance_res = client.post("/compute/v1/instances", {
    "name": "comfyui-flux-worker",
    "catalog_id": "g5.xlarge",
    "region": "eu-central-1",
    "spot_instance": True,                 # ~62% discount
    "max_runtime_hours": 8,                # Auto-stop after 8 hours
    "root_volume_type": "gp3",
    "root_volume_size_gb": 120,            
    "os_image": "ubuntu-2204-lts",
    "install_presets": ["docker", "python-ml", "nvidia-docker"],
    "startup_script": "#!/bin/bash
docker run -d --gpus all -p 8188:8188 --name comfyui --restart unless-stopped -v /data/models:/workspace/ComfyUI/models yanwk/comfyui-boot:latest",
    "security_group_rules": [
        {"protocol": "tcp", "port": 22, "cidr": "0.0.0.0/0", "description": "SSH Access"},
        {"protocol": "tcp", "port": 8188, "cidr": "0.0.0.0/0", "description": "ComfyUI Web Interface"}
    ]
})
print("Provisioned:", instance_res["instance"]["id"])

Scenario 2: High-Throughput vLLM OpenAI Server (Spot Instance)

Deploy high-throughput inference for open-weights models like Qwen2.5 with an OpenAI-compatible API endpoint.

cURL

bash

curl -X POST https://apis.fotohub.app/compute/v1/instances \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "vllm-openai-server",
    "catalog_id": "g5.2xlarge",
    "region": "eu-central-1",
    "spot_instance": true,
    "max_runtime_hours": 12,
    "root_volume_size_gb": 200,
    "install_presets": ["docker", "nvidia-docker"],
    "security_group_rules": [
      {"protocol": "tcp", "port": 22, "cidr": "0.0.0.0/0"},
      {"protocol": "tcp", "port": 8000, "cidr": "0.0.0.0/0"}
    ],
    "startup_script": "#!/bin/bash
docker run -d --gpus all -p 8000:8000 --ipc=host vllm/vllm-openai:latest --model Qwen/Qwen2.5-7B-Instruct --port 8000"
  }'

Once running, query it:

python

from openai import OpenAI
client = OpenAI(base_url="http://YOUR_INSTANCE_IP:8000/v1", api_key="none")
completion = client.chat.completions.create(
    model="Qwen/Qwen2.5-7B-Instruct",
    messages=[{"role": "user", "content": "Explain quantum computing in 3 sentences."}]
)
print(completion.choices[0].message.content)

Scenario 3: Cheap Background Inference on NVIDIA T4 (G4dn)

Use a cheaper g4dn.xlarge instance for tasks that don't need A10G speed.

TypeScript

typescript

const res = await client.post("/compute/v1/instances", {
  name: "cheap-t4-worker",
  catalog_id: "g4dn.xlarge", // NVIDIA T4, $0.20/hr spot
  spot_instance: true,
  max_runtime_hours: 24,
  root_volume_size_gb: 80,
  region: "eu-central-1",
  install_presets: ["python-ml"],
  startup_script: "pip install transformers torch; python -c 'import torch; print(torch.cuda.is_available())' > /tmp/cuda_ok.txt"
});

Scenario 4: Persistent EBS Model Cache Pattern

For large models (e.g., Llama-3-70B), repeatedly downloading 140GB is inefficient. Attach a persistent EBS volume.

cURL

bash

# 1. Create a persistent volume (assume endpoint exists)
# 2. Attach to instance on provision
curl -X POST https://apis.fotohub.app/compute/v1/instances \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" \
  -d '{
    "name": "persistent-model-cache",
    "catalog_id": "g5.xlarge",
    "region": "eu-central-1",
    "attached_volumes": [{"volume_id": "vol_abc123", "mount_path": "/mnt/models"}],
    "startup_script": "echo \"export HF_HOME=/mnt/models\" >> /etc/environment"
  }'

Scenario 5: Multi-AZ Fault Tolerant Deployment

Deploy across multiple Availability Zones in eu-central-1 to ensure high availability.

Python

python

zones = ["eu-central-1a", "eu-central-1b", "eu-central-1c"]
instances = []
for zone in zones:
    res = client.post("/compute/v1/instances", {
        "name": f"worker-{zone}",
        "catalog_id": "g4dn.xlarge",
        "region": "eu-central-1",
        "availability_zone": zone,
        "spot_instance": True
    })
    instances.append(res.json()["instance"]["id"])
print(f"Deployed to 3 AZs: {instances}")

Advanced Architecture

Custom AMIs

Instead of relying purely on install_presets and startup_script, you can snapshot a configured instance into a Custom AMI (Amazon Machine Image). Pass the AMI ID into the os_image field during provisioning to drastically reduce boot times from minutes to seconds.

json

{
  "catalog_id": "g5.xlarge",
  "os_image": "ami-0123456789abcdef0", 
  "region": "eu-central-1"
}

Multiple AZ Placement

Specify availability_zone explicitly (eu-central-1a, eu-central-1b, eu-central-1c) to distribute risk. If Spot capacity is constrained in 1a, fallback to 1b.

Remote Script Execution & Monitoring

Execute Ad-hoc Scripts (`POST /run-script`)

Execute ad-hoc maintenance or orchestration scripts without opening an interactive SSH session.

bash

curl -X POST https://apis.fotohub.app/compute/v1/instances/inst_9a8b7c/run-script \
  -H "Authorization: Bearer $FOTOHUB_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"script": "nvidia-smi --query-gpu=name,temperature.gpu,utilization.gpu,memory.total,memory.used --format=csv,noheader"}'

CloudWatch Metrics & Health Telemetry

Monitor hardware pressure, VRAM allocation, and network spikes in 5-minute aggregation buckets.

bash

curl -X GET "https://apis.fotohub.app/compute/v1/instances/inst_9a8b7c/metrics?period=300&hours=4" \
  -H "Authorization: Bearer $FOTOHUB_API_KEY"

Inspecting Serial Console Output

When debugging boot failures, kernel panics, or cloud-init startup scripts.

bash

curl -X GET "https://apis.fotohub.app/compute/v1/instances/inst_9a8b7c/logs?lines=50" \
  -H "Authorization: Bearer $FOTOHUB_API_KEY"

Comprehensive FAQ & Troubleshooting

Why is my instance stuck in `pending`?

Usually, this means AWS is allocating underlying hardware. If it remains stuck for over 5 minutes, the selected Availability Zone might be out of Spot capacity. Try deploying to a different AZ or use On-Demand.

How do I upgrade CUDA versions?

By default, the nvidia-cuda preset installs a stable, platform-tested version of CUDA (e.g., 12.1 or 12.2). To upgrade, you can execute a run-script command to purge existing drivers and pull the latest from NVIDIA's official apt repositories.

What happens when max_runtime_hours is reached?

The platform automatically sends a SIGTERM signal to your processes and gracefully terminates the instance. Any data stored on the root volume that is not backed up or pushed to a persistent storage service (like S3 or a detached EBS volume) will be permanently lost.

Can I attach multiple EBS volumes?

Yes. Use the attached_volumes array during provisioning. You must handle the OS-level mounting (e.g., mkfs.ext4 and mount /dev/nvme1n1 /data) via the startup_script.

Why does my SSH connection timeout?

Ensure you have a Security Group rule allowing TCP port 22 from your IP (or 0.0.0.0/0). Also, verify the instance status is running and a public IP has been assigned.

Do you offer reserved instances?

Currently, we only offer Spot and On-Demand pricing models. For custom enterprise commitments, contact our sales team.

Is IPv6 Supported?

At this time, instances are provisioned with IPv4 addresses only. Internal VPC routing supports standard private IPv4 CIDR blocks.

How does the prepaid wallet deduction work?

Your wallet is locked for the max_runtime_hours cost estimate. If you terminate the instance early, the unused portion is immediately refunded to your wallet balance. Billing is per-second after the first 60 seconds.

Deep Dive: Optimizing Inference Performance on A10G & T4

Maximizing throughput and minimizing latency requires tuning at the framework level.

vLLM Optimization Strategies

Tensor Parallelism: If you resize to an instance with multiple GPUs (e.g., g5.12xlarge with 4x A10G), enable tensor parallelism by adding -tp 4 to your vLLM command.
Quantization: Load models in AWQ or GPTQ formats to drastically reduce memory bandwidth bottlenecks.
Prefix Caching: Enable --enable-prefix-caching for workloads with highly repetitive system prompts.
PagedAttention Block Size: Tune the block size depending on your context length requirements.

ComfyUI Optimization Strategies

xformers: Ensure xformers is installed to enable memory-efficient attention.
FP8 Weights: For large models like Flux, load UNet weights in FP8 format to fit within the 24GB VRAM of an A10G.
Triton Kernels: For custom nodes, ensure PyTorch is compiled with Triton support for kernel fusion.

PyTorch Direct Optimization

Torch.compile: Utilize torch.compile(model, mode="reduce-overhead") in PyTorch 2.x to JIT compile your model graphs.
Mixed Precision: Always use torch.autocast(device_type='cuda', dtype=torch.bfloat16) on G5 instances, as A10G supports native bfloat16 for improved numerical stability over fp16.

Exhaustive API Error Codes Reference

When interacting with the apis.fotohub.app/compute/v1 endpoints, you may encounter HTTP error responses. Below is a comprehensive list of error codes, their meanings, and remediation steps.

HTTP Code	Error Code	Description	Remediation
400	`ValidationError`	The request payload is malformed or missing required fields.	Check your JSON schema. Ensure `catalog_id` and `region` are valid.
401	`Unauthorized`	Invalid or missing `FOTOHUB_API_KEY`.	Verify the token in the `Authorization: Bearer` header.
402	`PaymentRequired`	Wallet balance is below the $0.50 USD minimum.	Top up your wallet in the `/console` dashboard.
403	`Forbidden`	API key lacks permissions for this action.	Ensure your key has `compute:write` scopes.
404	`InstanceNotFound`	The requested instance ID does not exist or was terminated.	Check the instance ID. Terminated instances are purged from the database after 7 days.
409	`ConflictState`	Cannot perform action in the current lifecycle state (e.g., resizing a running instance).	Stop the instance before attempting a resize.
422	`CapacityUnavailable`	The requested Spot capacity is currently unavailable in the target AZ.	Try a different AZ or switch `spot_instance` to `false`.
429	`RateLimitExceeded`	Too many requests within a 1-minute window.	Implement exponential backoff in your polling loops. Standard limit is 60 req/min.
500	`InternalServerError`	An unexpected orchestration failure occurred.	Check our status page. Retry with a new provisioning request.
503	`ServiceUnavailable`	The compute backend is undergoing maintenance.	Retry later. Maintenance windows are announced via email.

Supported OS Images (`os_image`)

Image Tag	Base OS	Kernel Version	Architecture	Pre-installed Tools
`ubuntu-2204-lts`	Ubuntu 22.04 LTS	5.15+ (AWS optimized)	x86_64	curl, wget, git, awscli
`ubuntu-2004-lts`	Ubuntu 20.04 LTS	5.11+ (AWS optimized)	x86_64	curl, wget, git, awscli
`debian-12`	Debian 12 (Bookworm)	6.1+	x86_64	apt-transport-https, ca-certificates
`amazon-linux-2023`	Amazon Linux 2023	6.1+	x86_64	dnf, awscli pre-configured

GPU & CPU On-Demand Rental ​

GPU Family Catalog & Hardware Specifications ​

G5 Instances (NVIDIA A10G) ​

G4dn Instances (NVIDIA T4) ​

Spot vs On-Demand Decision ​

Billing & Cost Management ​

Key Billing Rules ​

Cost Calculation Formula ​

Lifecycle API Summary ​

Step-by-Step Guide: Provisioning a GPU Node ​

1. Check Eligibility ​

2. Estimate Costs ​

3. Provision the Instance ​

4. Wait for Running & Fetch IP (Polling Loop) ​

5. SSH Key Automation & Login ​

Lifecycle Operations Guide ​

Start, Stop, Reboot, Terminate ​

Dynamic Vertical Resizing (POST /resize) ​

Install Presets Reference ​

Security Group Rules Reference ​

Cloud-Init & ML Stack Examples ​

PyTorch & HuggingFace Setup ​

vLLM & DeepSeek Setup ​

5 Real-World Provisioning Scenarios ​

Scenario 1: Provisioning a ComfyUI Worker (NVIDIA A10G) ​

Scenario 2: High-Throughput vLLM OpenAI Server (Spot Instance) ​

Scenario 3: Cheap Background Inference on NVIDIA T4 (G4dn) ​

Scenario 4: Persistent EBS Model Cache Pattern ​

Scenario 5: Multi-AZ Fault Tolerant Deployment ​

Advanced Architecture ​

Custom AMIs ​

Multiple AZ Placement ​

Remote Script Execution & Monitoring ​

Execute Ad-hoc Scripts (POST /run-script) ​

CloudWatch Metrics & Health Telemetry ​

Inspecting Serial Console Output ​

Comprehensive FAQ & Troubleshooting ​

Why is my instance stuck in pending? ​

How do I upgrade CUDA versions? ​

What happens when max_runtime_hours is reached? ​

Can I attach multiple EBS volumes? ​

Why does my SSH connection timeout? ​

Do you offer reserved instances? ​

Is IPv6 Supported? ​

How does the prepaid wallet deduction work? ​

Deep Dive: Optimizing Inference Performance on A10G & T4 ​

vLLM Optimization Strategies ​

ComfyUI Optimization Strategies ​

PyTorch Direct Optimization ​

Exhaustive API Error Codes Reference ​

Supported OS Images (os_image) ​

GPU & CPU On-Demand Rental

GPU Family Catalog & Hardware Specifications

G5 Instances (NVIDIA A10G)

G4dn Instances (NVIDIA T4)

Spot vs On-Demand Decision

Billing & Cost Management

Key Billing Rules

Cost Calculation Formula

Lifecycle API Summary

Step-by-Step Guide: Provisioning a GPU Node

1. Check Eligibility

2. Estimate Costs

3. Provision the Instance

4. Wait for Running & Fetch IP (Polling Loop)

5. SSH Key Automation & Login

Lifecycle Operations Guide

Start, Stop, Reboot, Terminate

Dynamic Vertical Resizing (`POST /resize`)

Install Presets Reference

Security Group Rules Reference

Cloud-Init & ML Stack Examples

PyTorch & HuggingFace Setup

vLLM & DeepSeek Setup

5 Real-World Provisioning Scenarios

Scenario 1: Provisioning a ComfyUI Worker (NVIDIA A10G)

Scenario 2: High-Throughput vLLM OpenAI Server (Spot Instance)

Scenario 3: Cheap Background Inference on NVIDIA T4 (G4dn)

Scenario 4: Persistent EBS Model Cache Pattern

Scenario 5: Multi-AZ Fault Tolerant Deployment

Advanced Architecture

Custom AMIs

Multiple AZ Placement

Remote Script Execution & Monitoring

Execute Ad-hoc Scripts (`POST /run-script`)

CloudWatch Metrics & Health Telemetry

Inspecting Serial Console Output

Comprehensive FAQ & Troubleshooting

Why is my instance stuck in `pending`?

How do I upgrade CUDA versions?

What happens when max_runtime_hours is reached?

Can I attach multiple EBS volumes?

Why does my SSH connection timeout?

Do you offer reserved instances?

Is IPv6 Supported?

How does the prepaid wallet deduction work?

Deep Dive: Optimizing Inference Performance on A10G & T4

vLLM Optimization Strategies

ComfyUI Optimization Strategies

PyTorch Direct Optimization

Exhaustive API Error Codes Reference

Supported OS Images (`os_image`)