TensorZero Dart Client #

A full-featured, enterprise-grade Dart client for TensorZero Gateway supporting both OpenAI-compatible and native TensorZero APIs with streaming, tool calling, multimodal capabilities, and advanced features.

🚀 Features #

✅ Complete API Coverage #

• OpenAI-Compatible API (/openai/v1/chat/completions): Drop-in replacement for OpenAI clients
• Native TensorZero API (/inference): Full access to TensorZero's advanced features

🔧 Core Capabilities #

• Streaming Support: Real-time responses with Server-Sent Events
• Tool Calling: Function tools, OpenAI custom tools, provider tools
• Multimodal Content: Text, images, audio, and custom content blocks
• Caching: Read/write caching with configurable options
• Credentials: Dynamic credential injection
• Episodes: Request tracking and management

🏗️ Enterprise Architecture #

• SOLID Design: Clean, maintainable, extensible code
• Functional Programming: Result types for error handling
• Type Safety: Comprehensive null safety and union types
• Clean Architecture: Clear separation of concerns

🧪 Quality Assurance #

• Comprehensive Testing: Unit, integration, and contract tests
• Error Resilience: Robust error handling and recovery
• Performance: Optimized serialization and streaming
• Documentation: Complete API docs and examples

📦 Installation #

dependencies:
  tensorzero:
    git:
      url: https://github.com/tensorzero/tensorzero
      path: clients/dart

🚀 Quick Start #

Simple OpenAI-Compatible Client #

import 'package:tensorzero/tensorzero.dart';

// Create client
final client = SimpleOpenAIClient(
  baseUrl: 'http://localhost:3000',
);

// OpenAI-compatible chat completion
final response = await client.createChatCompletionRaw(
  model: 'tensorzero::model_name::your-model',
  messages: [
    {'role': 'user', 'content': 'Hello, world!'}
  ],
  temperature: 0.7,
  maxTokens: 100,
);

// Handle response
response.fold(
  (data) => print('Response: ${data['choices'][0]['message']['content']}'),
  (error) => print('Error: $error'),
);

Native TensorZero Inference #

import 'package:tensorzero/tensorzero.dart';

// Create client
final client = SimpleOpenAIClient(
  baseUrl: 'http://localhost:3000',
);

// Native TensorZero inference
final inference = await client.createInferenceRaw(
  modelName: 'your-model',
  input: {
    'messages': [
      {
        'role': 'user',
        'content': [
          {'type': 'text', 'text': 'Hello!'}
        ]
      }
    ]
  },
);

// Handle response
inference.fold(
  (data) {
    final content = data['content'] as List;
    for (final block in content) {
      if (block['type'] == 'text') {
        print('Response: ${block['text']}');
      }
    }
  },
  (error) => print('Error: $error'),
);

🚀 Quick Start #

OpenAI-Compatible Client #

import 'package:tensorzero/tensorzero.dart';

// Create client
final client = TensorZeroOpenAIClient(
  baseUrl: 'http://localhost:3000',
  config: BaseClientConfig(
    baseUrl: 'http://localhost:3000',
    apiKey: 'your-api-key', // optional
  ),
);

// Basic chat completion
final response = await client.createChatCompletion(
  model: 'tensorzero::function_name::chat_completion',
  messages: [
    OpenAIMessage(role: 'user', content: 'Hello, world!'),
  ],
  temperature: 0.7,
);

// Handle response
response.fold(
  (response) => print('Response: ${response.choices.first.message.content}'),
  (error) => print('Error: $error'),
);

// Streaming
final stream = client.createChatCompletionStream(
  model: 'tensorzero::function_name::chat_completion',
  messages: [OpenAIMessage(role: 'user', content: 'Tell me a story')],
);

await for (final chunk in stream) {
  chunk.fold(
    (chunk) => print(chunk.choices.first.delta?.content ?? ''),
    (error) => print('Error: $error'),
  );
}

Native TensorZero Client #

import 'package:tensorzero/tensorzero.dart';

// Create native client
final client = TensorZeroClient(
  baseUrl: 'http://localhost:3000',
  config: BaseClientConfig(baseUrl: 'http://localhost:3000'),
);

// Inference with function
final response = await client.inference(
  functionName: 'chat_completion',
  input: InferenceInput(
    messages: [
      TensorZeroMessage(
        role: 'user',
        content: [TextContentBlock(text: 'Hello!')],
      ),
    ],
  ),
  cacheOptions: CacheOptions.writeOnly(),
);

// Handle complex content blocks
response.fold(
  (response) {
    for (final block in response.content as List<ContentBlock>) {
      block.map(
        text: (block) => print('Text: ${block.text}'),
        toolCall: (block) => print('Tool: ${block.name}'),
        file: (block) => print('File: ${block.filename}'),
        // ... handle all content types
      );
    }
  },
  (error) => print('Error: $error'),
);

🛠️ Advanced Features #

Tool Calling #

// Define tools
final tools = [
  OpenAITool(
    type: 'function',
    function: FunctionToolDefinition(
      name: 'get_weather',
      description: 'Get weather for a location',
      parameters: {
        'type': 'object',
        'properties': {
          'location': {'type': 'string'},
          'unit': {'type': 'string', 'enum': ['celsius', 'fahrenheit']},
        },
        'required': ['location'],
      },
    ),
  ),
];

// Use with OpenAI client
final response = await client.createChatCompletion(
  model: 'tensorzero::function_name::weather_bot',
  messages: [OpenAIMessage(role: 'user', content: 'Weather in Tokyo?')],
  tools: tools,
  toolChoice: 'auto',
);

// Handle tool calls
final message = response.choices.first.message;
for (final toolCall in message.toolCalls ?? []) {
  print('Calling: ${toolCall.function.name}');
  print('Args: ${toolCall.function.arguments}');
}

Multimodal Content #

// Images and files
final response = await tzClient.inference(
  functionName: 'multimodal_assistant',
  input: InferenceInput(
    messages: [
      TensorZeroMessage(
        role: 'user',
        content: [
          TextContentBlock(text: 'Describe this image'),
          FileContentBlock.fileFromUrl(
            url: 'https://example.com/image.jpg',
            detail: 'high',
          ),
          FileContentBlock.fileFromBase64(
            data: base64ImageData,
            mimeType: 'image/png',
          ),
        ],
      ),
    ],
  ),
);

TensorZero-Specific Features #

// Advanced configuration
final response = await client.createChatCompletion(
  model: 'tensorzero::function_name::advanced_function',
  messages: [OpenAIMessage(role: 'user', content: 'Hello!')],

  // Episode tracking
  tensorzeroEpisodeId: 'episode-123',

  // Caching
  tensorzeroCacheOptions: CacheOptions.enabled(maxAgeSeconds: 3600),

  // Dynamic credentials
  tensorzeroCredentials: {'api_key': 'secret'},

  // Provider tools
  tensorzeroProviderTools: [
    ProviderTool(
      scope: ProviderToolScope(modelName: 'gpt-4'),
      tool: {'type': 'web_search'},
    ),
  ],

  // Extra headers/body modifications
  tensorzeroExtraHeaders: [
    ExtraHeaderModification(
      name: 'X-Custom-Header',
      value: 'custom-value',
    ),
  ],

  // Tags for observability
  tensorzeroTags: {'user_id': '12345', 'priority': 'high'},
);

🏛️ Architecture #

Clean Architecture Layers #

┌─────────────────────────────────────┐
│         Presentation Layer          │  ← Client APIs
│   (OpenAI Client, TensorZero Client) │
├─────────────────────────────────────┤
│         Application Layer           │  ← Use Cases, DTOs
│      (Request/Response Models)      │
├─────────────────────────────────────┤
│         Domain Layer                │  ← Business Logic
│    (Entities, Services, Validators) │
├─────────────────────────────────────┤
│       Infrastructure Layer          │  ← External Concerns
│   (HTTP, Serialization, Streaming)  │
└─────────────────────────────────────┘

Key Components #

• Core: Error handling, result types, validation
• Infrastructure: HTTP client, retry logic, streaming
• Domain: Business entities, value objects, services
• Application: DTOs, use cases, mappers
• Presentation: Client APIs, builders, configuration

🔧 Configuration #

Client Configuration #

final config = BaseClientConfig(
  baseUrl: 'http://localhost:3000',
  apiKey: 'your-api-key',
  timeoutConfig: TimeoutPresets.standard,
  retryPolicy: RetryPolicies.production,
  userAgent: 'MyApp/1.0.0',
);

Timeout Presets #

// Fast timeouts for development
TimeoutPresets.fast

// Standard production timeouts
TimeoutPresets.standard

// Slow timeouts for complex operations
TimeoutPresets.slow

// Custom timeouts
TimeoutConfig(
  requestTimeout: Duration(seconds: 60),
  streamingTimeout: Duration(minutes: 5),
)

Retry Policies #

// Development (quick retries)
RetryPolicies.development

// Production (conservative retries)
RetryPolicies.production

// Custom retry policy
CustomRetryPolicy(
  maxAttempts: 5,
  delayCalculator: (attempt) => Duration(seconds: attempt * 2),
  retryableErrorPredicate: (error) => error is NetworkException,
)

🧪 Testing #

Unit Tests #

dart test test/unit/

Integration Tests #

dart test test/integration/

Mock Setup #

import 'package:mockito/mockito.dart';

class MockHttpClient extends Mock implements HttpClient {}
class MockTensorZeroClient extends Mock implements TensorZeroClient {}

📚 API Reference #

OpenAI-Compatible Client #

TensorZeroOpenAIClient

• createChatCompletion() - Synchronous chat completion
• createChatCompletionStream() - Streaming chat completion

Native TensorZero Client #

TensorZeroClient

• inference() - Synchronous inference
• inferenceStream() - Streaming inference

Content Block Types #

Text Content

TextContentBlock(text: 'Hello', arguments: {'key': 'value'})

Tool Calls/Results

ToolCallContentBlock(id: '123', name: 'func', arguments: {...})
ToolResultContentBlock(id: '123', name: 'func', result: 'output')

Files (Multimodal)

FileContentBlock.fileFromUrl(url: '...', detail: 'high')
FileContentBlock.fileFromBase64(data: '...', mimeType: 'image/png')

Custom Content

RawTextContentBlock(value: 'unprocessed text')
TemplateContentBlock(name: 'template', arguments: {...})
ThoughtContentBlock(text: 'reasoning...')
UnknownContentBlock(data: {...}, modelName: 'gpt-4')

🚀 Performance & Reliability #

Streaming Optimization #

• Minimal memory usage for large responses
• Efficient SSE parsing
• Configurable buffer sizes

Error Recovery #

• Automatic retries with exponential backoff
• Circuit breaker pattern
• Graceful degradation

Connection Management #

• HTTP/2 support where available
• Connection pooling
• DNS caching

🔒 Security #

Credential Management #

• Secure credential storage
• Dynamic credential injection
• Credential rotation support

Input Validation #

• Comprehensive input sanitization
• JSON Schema validation
• Type-safe parameter handling

Network Security #

• TLS certificate validation
• Secure header handling
• Request/response logging (configurable)

📈 Monitoring & Observability #

Metrics Collection #

// Automatic metrics
- Request latency
- Error rates
- Token usage
- Cache hit rates

Logging #

// Configurable logging levels
Logger.root.level = Level.INFO;

Tracing #

// Integration with OpenTelemetry
tensorzeroOtlpTracesExtraHeaders: {
  'user.id': '12345',
  'request.id': 'abc-123',
}

🤝 Contributing #

Development Setup #

# Clone repository
git clone https://github.com/tensorzero/tensorzero.git
cd tensorzero/clients/dart

# Install dependencies
dart pub get

# Run tests
dart test

# Generate code
dart run build_runner build

# Run example
dart run example/tensorzero_example.dart

Code Quality #

# Lint code
dart analyze

# Format code
dart format .

# Run tests with coverage
dart test --coverage

📄 License #

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

🙏 Acknowledgments #

• TensorZero for the gateway
• Dart for the language
• Freezed for union types
• HTTP for networking

Built with ❤️ for the TensorZero community

Enterprise-grade Dart client for production TensorZero deployments