Sources / External Identifiers

Sources are a fundamental concept in the Propeller REST API that enable seamless integration with external systems like ERP, CRM, and other business applications. They provide a way to maintain unique identifiers from external systems while ensuring data integrity and preventing conflicts.

What are Sources?

Sources represent the origin system or application that provides data to Propeller. They consist of two key components:

1. source - The name or identifier of the external system (e.g., "MBC", "EXACTGLOBE", "REST-API")
2. sourceId - The unique identifier from that external system

The combination of source + sourceId creates a unique identifier that allows Propeller to:

• Track where data originated
• Prevent conflicts when the same ID exists in multiple systems
• Enable efficient bulk operations
• Maintain data lineage and audit trails

Why Sources are Important

1. Multi-System Integration

When integrating with multiple external systems, the same ID might exist in different systems. Sources prevent conflicts:

// Microsoft Business Central customer
{
  "sourceId": "12345",
  "source": "MBC",
  "name": "Company ABC"
}

// Exact Globe customer (same ID, different source)
{
  "sourceId": "12345", 
  "source": "EXACTGLOBE",
  "name": "Company XYZ"
}

2. Efficient Bulk Operations

Sources enable efficient bulk create/update operations by using the sourceId + source combination as lookup keys:

{
  "usergroups": [
    {
      "sourceId": "x-123",
      "source": "MBC",
      "name": "Company X"
    },
    {
      "sourceId": "y-456", 
      "source": "MBC",
      "name": "Company Y"
    }
  ]
}

3. Data Lineage and Auditing

Sources provide traceability back to the original system, making it easier to:

• Debug data issues
• Audit data changes
• Understand data flow between systems

Source Representation Patterns

The Propeller REST API uses two different patterns for representing sources, depending on the context:

Pattern 1: Direct Source Fields

Most entities use direct source and sourceId fields:

{
  "id": 12345,
  "name": "John Doe",
  "email": "john@example.com",
  "source": "REST-API",
  "sourceId": "3002196"
}

Used in:

• Users
• Usergroups
• Customers
• Contacts
• Companies
• Products
• Categories
• Clusters
• And most other entities

Pattern 2: Source Objects

Some entities use a source object with name and id properties:

{
  "productSource": {
    "name": "Source",
    "id": "Source 1"
  }
}

Used in:

• Order items (productSource)
• Company references (companySource)

Common Source Values

Source Name	Description	Typical Use Case
REST-API	Data created via REST API	Manual data entry, API integrations
MBC	Microsoft Business Central	ERP integration
EXACTGLOBE	Exact Globe ERP	ERP integration
TECHDATA	TechData supplier system	Supplier data integration
RCEB0451-3	Custom ERP system	Internal ERP integration
OTTEVANGER-TRIMERGO	Specific business system	Business-specific integration

Using Sources in API Operations

1. Creating Resources with Sources

When creating resources, you can specify the source to maintain external system references:

POST /v2/users
{
  "firstName": "John",
  "lastName": "Doe", 
  "email": "john@example.com",
  "source": "MBC",
  "sourceId": "3002196",
  "parent": {
    "sourceId": "x-123",
    "source": "MBC"
  }
}

2. Bulk Operations with Sources

Bulk endpoints use sources to determine whether to create or update resources:

POST /v2/users/bulk/sourceId
{
  "users": [
    {
      "source": "MBC",
      "sourceId": "3002196",
      "firstName": "John",
      "lastName": "Doe",
      "email": "john@example.com",
      "parent": {
        "sourceId": "x-123",
        "source": "MBC"
      }
    }
  ]
}

3. Lookup Operations with Sources

You can retrieve resources using their source identifiers:

GET /v2/users/sourceId/3002196?source=MBC

4. Parent References with Sources

When creating child resources, reference parents using their source information:

{
  "source": "MBC",
  "sourceId": "3002197",
  "parent": {
    "sourceId": "x-123",
    "source": "MBC"
  },
  "firstName": "Jane",
  "lastName": "Smith"
}

Source Validation and Constraints

Required Fields

• When sourceId is provided, source is required
• When source is provided, sourceId is required
• The combination must be unique within the tenant

Naming Conventions

• Source names should be descriptive and consistent
• Use uppercase for system names (e.g., "MBC", "EXACTGLOBE")
• Avoid special characters that might cause URL encoding issues

Best Practices

1. Consistent Naming: Use the same source name across all related entities
2. Documentation: Document your source naming conventions
3. Validation: Validate source + sourceId combinations before sending to API
4. Error Handling: Handle cases where source combinations might conflict

Examples by Entity Type

Users and Usergroups

// Creating a usergroup with source
{
  "name": "Company ABC",
  "sourceId": "x-123",
  "source": "MBC",
  "parent": {
    "sourceId": "999112000",
    "source": "MBC"
  }
}

// Creating a user within that usergroup
{
  "firstName": "John",
  "lastName": "Doe",
  "source": "MBC", 
  "sourceId": "3002196",
  "parent": {
    "sourceId": "x-123",
    "source": "MBC"
  }
}

Products

{
  "name": "Product ABC",
  "sku": "ABC123",
  "source": "TECHDATA",
  "sourceId": "NCABD70004",
  "status": "A"
}

Orders with Product Sources

{
  "items": [
    {
      "name": "Product XYZ",
      "sku": "XYZ789",
      "productSource": {
        "name": "TECHDATA",
        "id": "NCABD70004"
      },
      "quantity": 2,
      "price": 29.99
    }
  ]
}

Error Handling

Common source-related errors:

Error Code	Description	Solution
10006	Invalid lookup key	Ensure you're using a valid lookup key (id, sourceId)
70004	User already exists	Check if user with same source + sourceId exists
70005	Multiple users found	Provide additional filters to narrow down results

Migration and Data Import Strategies

1. Initial Data Import

When importing data from external systems:

1. Map external system identifiers to source names
2. Use bulk endpoints for efficiency
3. Maintain parent-child relationships using sources

2. Ongoing Synchronization

For continuous data synchronization:

1. Use source + sourceId as the primary lookup mechanism
2. Implement delta updates to minimize API calls
3. Handle conflicts gracefully

3. Data Cleanup

When cleaning up or migrating data:

1. Use sources to identify data origin
2. Implement proper validation before deletion
3. Maintain audit trails

Conclusion

Sources are essential for building robust integrations with external systems. They provide:

• Uniqueness: Prevent ID conflicts across systems
• Traceability: Track data origin and lineage
• Efficiency: Enable fast bulk operations
• Flexibility: Support multiple external systems simultaneously

By understanding and properly implementing sources, you can build reliable, scalable integrations that maintain data integrity across your entire business ecosystem.