Skip to main content

Sources / External Identifiers

Sources are a fundamental concept in the Propeller REST API that enable seamless integration with external systems like ERP, CRM, and other business applications. They provide a way to maintain unique identifiers from external systems while ensuring data integrity and preventing conflicts.

What are Sources?​

Sources represent the origin system or application that provides data to Propeller. They consist of two key components:

  1. source - The name or identifier of the external system (e.g., "MBC", "EXACTONLINE", "REST-API")
  2. sourceId - The unique identifier from that external system

The combination of source + sourceId creates a unique identifier that allows Propeller to:

  • Track where data originated
  • Prevent conflicts when the same ID exists in multiple systems
  • Enable efficient bulk operations
  • Maintain data lineage and audit trails

Why Sources are Important​

1. Multi-System Integration​

When integrating with multiple external systems, the same ID might exist in different systems. Sources prevent conflicts:

// Microsoft Business Central customer
{
  "sourceId": "12345",
  "source": "MBC",
  "name": "Company ABC"
}

// Exact Online customer (same ID, different source)
{
  "sourceId": "12345", 
  "source": "EXACTONLINE",
  "name": "Company XYZ"
}

2. Efficient Bulk Operations​

Sources enable efficient bulk create/update operations by using the sourceId + source combination as lookup keys:

{
  "companies": [
    {
      "sourceId": "x-123",
      "source": "MBC",
      "name": "Company X"
    },
    {
      "sourceId": "y-456", 
      "source": "MBC",
      "name": "Company Y"
    }
  ]
}

3. Data Lineage and Auditing​

Sources provide traceability back to the original system, making it easier to:

  • Debug data issues
  • Audit data changes
  • Understand data flow between systems

Source Representation Patterns​

The Propeller REST API uses two different patterns for representing sources, depending on the context:

Pattern 1: Direct Source Fields​

Most entities use direct source and sourceId fields in the create endpoints:

{
  "id": 12345,
  "name": "John Doe",
  "email": "john@example.com",
  "source": "REST-API",
  "sourceId": "3002196"
}

But, in the get endpoints sources array of source objects is returned:

{
  "id": 12345,
  "name": "John Doe",
  "email": "john@example.com",
  "sources": [
      {
          "source": "REST-API",
          "sourceId": "3002196"
      }
  ],
}

Used in:

  • Users
  • Usergroups
  • Customers
  • Contacts
  • Companies
  • Products
  • Categories
  • Clusters
  • And most other entities

Pattern 2: Source Objects​

Some entities use a source object with name and id properties:

{
  "productSource": {
    "name": "REST-API",
    "id": "T551284"
  }
}

Used in:

  • Order items (productSource)
  • Company references (companySource)
important

The combination of source and sourceId fields is a legacy pattern from V1 that is being phased out in favor of a more structured approach. In REST API v2.0, we introduced a new source object structure with fields name and id, that provides better organization and extensibility.

The source object structure (name and id) will replace the source and sourceId fields across all entities.

Using Sources in API Operations​

1. Creating Resources with Sources​

When creating resources, you can specify the source to maintain external system references:

POST /v2/contacts
{
  "firstName": "John",
  "lastName": "Doe", 
  "email": "john@example.com",
  "source": "MBC",
  "sourceId": "3002196"
}

2. Bulk Operations with Sources​

Bulk endpoints use sources to determine whether to create or update resources:

POST /v2/users/bulk/sourceId
{
  "contacts": [
    {
      "source": "MBC",
      "sourceId": "3002196",
      "firstName": "John",
      "lastName": "Doe",
      "email": "john@example.com"
    }
  ]
}

3. Lookup Operations with Sources​

You can retrieve resources using their source identifiers:

GET /v2/contacts/sourceId/3002196?source=MBC

4. Parent References with Sources​

When creating child resources, reference parents using their source information:

{
  "source": "MBC",
  "sourceId": "3002197",
  "parent": {
    "sourceId": "x-123",
    "source": "MBC"
  },
  "firstName": "Jane",
  "lastName": "Smith"
}

Source Validation and Constraints​

Required Fields​

  • When sourceId is provided, source is required
  • When source is provided, sourceId is required
  • The combination must be unique within the tenant

Naming Conventions​

  • Source names should be descriptive and consistent
  • Use uppercase for system names (e.g., "MBC", "EXACTONLINE")
  • Avoid special characters that might cause URL encoding issues

Best Practices​

  1. Consistent Naming: Use the same source name across all related entities
  2. Documentation: Document your source naming conventions
  3. Validation: Validate source + sourceId combinations before sending to API
  4. Error Handling: Handle cases where source combinations might conflict

Conclusion​

Sources are essential for building robust integrations with external systems. They provide:

  • Uniqueness: Prevent ID conflicts across systems
  • Traceability: Track data origin and lineage
  • Efficiency: Enable fast bulk operations
  • Flexibility: Support multiple external systems simultaneously

By understanding and properly implementing sources, you can build reliable, scalable integrations that maintain data integrity across your entire business ecosystem.