`job.proto`

path mgmt/v1alpha1/job.proto

package mgmt.v1alpha1

Messages

`ActivityFailure`

Name	Type	Description
`message`	`string`	The message of the failure

`ActivityOptions`

Config that contains various timeouts that are configured in the underlying temporal workflow(s) and activities

Name	Type	Description
`schedule_to_close_timeout`	optional `int64`	Total time that a workflow is willing to wait for an activity to complete, including retries. Measured in seconds
`start_to_close_timeout`	optional `int64`	Max time of a single Temporal Activity execution attempt. This timeout should be as short as the longest psosible execution of any activity (e.g. table sync). Important to know that this is per retry attempt. Defaults to the schedule to close timeout if not provided. Measured in seconds
`retry_policy`	`RetryPolicy`	Optionally define a retry policy for the activity If max attempts is not set, the activity will retry indefinitely until the start to close timeout lapses

`AiGenerateSourceOptions`

Name	Type	Description
`ai_connection_id`	`string`	The connection id that corresponds with an AI-based Vydon connection
`schemas`	repeated `AiGenerateSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used to generate data for.
`fk_source_connection_id`	optional `string`	An optional connection id that will be used as the basis for the shape of data to be generated.
`model_name`	`string`	The name of the model to use
`user_prompt`	optional `string`	Optionally provide a user prompt to give more context to the schema
`generate_batch_size`	optional `int64`	The batch size that will be used to generate X number of records. This is global and will be applied to all tables configured.

`AiGenerateSourceSchemaOption`

Name	Type	Description
`schema`	`string`	The dataabase schema
`tables`	repeated `AiGenerateSourceTableOption`	The list of tables (and their configuration) that reside within the schema to receive generated data

`AiGenerateSourceTableOption`

Name	Type	Description
`table`	`string`	The table that will be used to generate data for
`row_count`	`int64`	The total number of records to be generated.

`AwsS3DestinationConnectionOptions`

Name	Type	Description
`storage_class`	`AwsS3DestinationConnectionOptions.StorageClass`	The storage class that will be used when objects are written to S3
`max_in_flight`	optional `uint32`	The maximum number of batched messages to have in flight at a given time. Increase this to improve throughput.
`timeout`	optional `string`	The maximum period (duration string) to wait on an upload before abandoning it and reattempting.
`batch`	`BatchConfig`	Configure batching options to more efficiently store records in S3

`AwsS3SourceConnectionOptions`

Name	Type	Description
`connection_id`	`string`	The unique connection id to a aws s3 connection configuration

`BatchConfig`

Name	Type	Description
`count`	optional `uint32`	The max allowed in a batch before it is flushed. 0 to disable.
`period`	optional `string`	A duration string in which an incomplete batch should be flushed regardless of the count. Examples are 1s, 1m, 500ms

`CancelJobRunRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account

`CancelJobRunResponse`

Name	Type	Description

`ColumnError`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table of the column
`column`	`string`	The column of the error
`errors`	repeated `string`	@deprecated - Use error_reports instead
`error_reports`	repeated `ColumnError.ColumnErrorReport`	The list of error reports

`ColumnError.ColumnErrorReport`

Column error report

Name	Type	Description
`code`	`ColumnError.ColumnErrorCode`	The error code
`message`	`string`	The error message

`ColumnWarning`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table of the column
`column`	`string`	The column of the warning
`warnings`	repeated `string`	@deprecated - Use warning_reports instead
`warning_reports`	repeated `ColumnWarning.ColumnWarningReport`	The list of warning reports

`ColumnWarning.ColumnWarningReport`

Column warning report

Name	Type	Description
`code`	`ColumnWarning.ColumnWarningCode`	The warning code
`message`	`string`	The warning message

`CreateJobDestination`

Name	Type	Description
`connection_id`	`string`	The connection id to use for the job destination
`options`	`JobDestinationOptions`	The destination options to use for the job destination

`CreateJobDestinationConnectionsRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job
`destinations`	repeated `CreateJobDestination`	The list of destinations to create and associate with the job

`CreateJobDestinationConnectionsResponse`

Name	Type	Description
`job`	`Job`	The job that was updated

`CreateJobHookRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job
`hook`	`NewJobHook`	The new hook configuration

`CreateJobHookResponse`

Name	Type	Description
`hook`	`JobHook`	The newly created hook

`CreateJobRequest`

Name	Type	Description
`account_id`	`string`	The unique account identifier that this job will be associated with
`job_name`	`string`	The unique, friendly name of the job. This is unique per account
`cron_schedule`	optional `string`	Optionally provide a cron schedule. Goes into effect if the job status is set to enabled
`mappings`	repeated `JobMapping`	The list of mappings that will be used to transform the data
`source`	`JobSource`	The source connection configuration
`destinations`	repeated `CreateJobDestination`	The list of destinations that will be used to store the data
`initiate_job_run`	`bool`	Initially trigger a run of this job regardless of its status or cron schedule
`workflow_options`	`WorkflowOptions`	Specify timeouts and other workflow options for the underlying temporal workflow
`sync_options`	`ActivityOptions`	Specify timeout and retry options for data synchronization activities Data sync activities are any piece of work that involves actually synchronizing data from a source to a destination For the data sync and generate jobs, this will be applied per table
`virtual_foreign_keys`	repeated `VirtualForeignConstraint`	The list of virtual foreign keys that will be used to further constrain the data ontop of the database defined constraints
`job_type`	optional `JobTypeConfig`	The type of job to create

`CreateJobResponse`

Name	Type	Description
`job`	`Job`	The job that was created

`CreateJobRunRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job

`CreateJobRunResponse`

Name	Type	Description

`DatabaseError`

Name	Type	Description
`errors`	repeated `string`	@deprecated - Use error_reports instead
`error_reports`	repeated `DatabaseError.DatabaseErrorReport`	The list of error reports

`DatabaseError.DatabaseErrorReport`

Database error report

Name	Type	Description
`code`	`DatabaseError.DatabaseErrorCode`	The error code
`message`	`string`	The error message

`DeleteJobDestinationConnectionRequest`

Name	Type	Description
`destination_id`	`string`	The unique identifier of the destination to delete

`DeleteJobDestinationConnectionResponse`

Name	Type	Description

`DeleteJobHookRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the hook

`DeleteJobHookResponse`

Name	Type	Description

`DeleteJobRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job to delete

`DeleteJobResponse`

Name	Type	Description

`DeleteJobRunRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account

`DeleteJobRunResponse`

Name	Type	Description

`DynamoDBDestinationConnectionOptions`

Configuration for DynamoDB Destination Connection Job Options

Name	Type	Description
`table_mappings`	repeated `DynamoDBDestinationTableMapping`	List of table mappings when piping data from a dynamoDB table to another dynamoDB table

`DynamoDBDestinationTableMapping`

Configuration for mapping a source table to a destination table for DynamoDB

Name	Type	Description
`source_table`	`string`	The name of the incoming source table
`destination_table`	`string`	The name of the outgoing destination table

`DynamoDBSourceConnectionOptions`

DynamoDB connection options for a job source

Name	Type	Description
`connection_id`	`string`	The unique connection id to a dynamodb connection configuration
`tables`	repeated `DynamoDBSourceTableOption`	List of table option configurations for any mapped source table. Any table listed in this must also be present as a job mapping table to be applied.
`unmapped_transforms`	`DynamoDBSourceUnmappedTransformConfig`	Default transformations for any unmapped keys
`enable_consistent_read`	`bool`	Enforces strong read consistency False: Eventually Consistent Reads, True: Strongly Consistent Reads https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/HowItWorks.ReadConsistency.html

`DynamoDBSourceSchemaSubset`

Name	Type	Description
`tables`	repeated `DynamoDBSourceTableOption`	The list of tables (and their configuration) that reside within the schema

`DynamoDBSourceTableOption`

Name	Type	Description
`table`	`string`	The table that this configuration will be applied to
`where_clause`	optional `string`	An optional PartiQL query that may be used for subsetting the DynamoDB table. This is not a parameterized query and must be valid. Intended to be everything after the WHERE keyword.

`DynamoDBSourceUnmappedTransformConfig`

Name	Type	Description
`b`	`JobMappingTransformer`	Byte
`boolean`	`JobMappingTransformer`	Boolean
`n`	`JobMappingTransformer`	Number
`s`	`JobMappingTransformer`	String

`GcpCloudStorageDestinationConnectionOptions`

Configuration for Google Cloud Storage Destination Connection Job Options

Name	Type	Description

`GenerateSourceOptions`

Name	Type	Description
`schemas`	repeated `GenerateSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used to generate data for.
`fk_source_connection_id`	optional `string`	An optional connection id that will be used as the basis for the shape of data to be generated.

`GenerateSourceSchemaOption`

Name	Type	Description
`schema`	`string`	The database schema
`tables`	repeated `GenerateSourceTableOption`	The list of tables (and their configuration) that reside within the schema to receive generated data

`GenerateSourceTableOption`

Name	Type	Description
`table`	`string`	The table that will be used to generate data for.
`row_count`	`int64`	The total number of records to be generated.

`GetActiveJobHooksByTimingRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job
`timing`	`GetActiveJobHooksByTimingRequest.Timing`	The timing desired.

`GetActiveJobHooksByTimingResponse`

Name	Type	Description
`hooks`	repeated `JobHook`	The active job hooks

`GetJobHookRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the hook

`GetJobHookResponse`

Name	Type	Description
`hook`	`JobHook`	The found hook

`GetJobHooksRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job

`GetJobHooksResponse`

Name	Type	Description
`hooks`	repeated `JobHook`	The list of hooks found attached to the job

`GetJobNextRunsRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job

`GetJobNextRunsResponse`

Name	Type	Description
`next_runs`	`JobNextRuns`	The list of next run times

`GetJobRecentRunsRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job

`GetJobRecentRunsResponse`

Name	Type	Description
`recent_runs`	repeated `JobRecentRun`	The list of recent job runs

`GetJobRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job

`GetJobResponse`

Name	Type	Description
`job`	`Job`	The job that was retrieved

`GetJobRunEventsRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account

`GetJobRunEventsResponse`

Name	Type	Description
`events`	repeated `JobRunEvent`	The list of events
`is_run_complete`	`bool`	Whether the run is complete

`GetJobRunLogsRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account
`window`	`LogWindow`	The time window in which to retrieve the logs
`max_log_lines`	optional `int64`	Optionally provide a max log limit
`log_levels`	repeated `LogLevel`	Provide a list of log levels to filter by. If any of these are UNSPECIFIED, all log levels are returned.

`GetJobRunLogsResponse`

Name	Type	Description
`log_lines`	repeated `GetJobRunLogsResponse.LogLine`	The list of log lines

`GetJobRunLogsResponse.LogLine`

Name	Type	Description
`log_line`	`string`	The log line
`timestamp`	optional `google.protobuf.Timestamp`	The timestamp of the log line
`labels`	repeated `GetJobRunLogsResponse.LogLine.LabelsEntry`	The labels associated with the log line

`GetJobRunLogsResponse.LogLine.LabelsEntry`

Name	Type	Description
`key`	`string`
`value`	`string`

`GetJobRunLogsStreamRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account
`window`	`LogWindow`	The time window in which to retrieve the logs
`should_tail`	`bool`	Whether or not to tail the stream. Note: only works with k8s-pods and is not currently supported with Loki logs
`max_log_lines`	optional `int64`	Optionally provide a max log limit
`log_levels`	repeated `LogLevel`	Provide a list of log levels to filter by. If any of these are UNSPECIFIED, all log levels are returned.

`GetJobRunLogsStreamResponse`

Name	Type	Description
`log_line`	`string`	The log line
`timestamp`	optional `google.protobuf.Timestamp`	The timestamp of the log line
`labels`	repeated `GetJobRunLogsStreamResponse.LabelsEntry`	The labels associated with the log line

`GetJobRunLogsStreamResponse.LabelsEntry`

Name	Type	Description
`key`	`string`
`value`	`string`

`GetJobRunRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account

`GetJobRunResponse`

Name	Type	Description
`job_run`	`JobRun`	The job run that was retrieved

`GetJobRunsRequest`

Name	Type	Description
`job_id`	`string`	Retireve runs for a specific job
`account_id`	`string`	Retrieve runs for all jobs in an account

`GetJobRunsResponse`

Name	Type	Description
`job_runs`	repeated `JobRun`	The list of job runs

`GetJobStatusRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job

`GetJobStatusResponse`

Name	Type	Description
`status`	`JobStatus`	The status of the job

`GetJobStatusesRequest`

Name	Type	Description
`account_id`	`string`	The unique identifier of the account

`GetJobStatusesResponse`

Name	Type	Description
`statuses`	repeated `JobStatusRecord`	The list of job statuses

`GetJobsRequest`

Name	Type	Description
`account_id`	`string`	The unique identifier of the account to get jobs for

`GetJobsResponse`

Name	Type	Description
`jobs`	repeated `Job`	The list of jobs

`GetPiiDetectionReportRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account

`GetPiiDetectionReportResponse`

Name	Type	Description
`report`	`PiiDetectionReport`	The pii detection report

`GetRunContextRequest`

Name	Type	Description
`id`	`RunContextKey`	The run context key

`GetRunContextResponse`

Name	Type	Description
`value`	`bytes`	The returned value in bytes. The format is determined by the key when it is set.

`IsJobHookNameAvailableRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job
`name`	`string`	The hook name to check.

`IsJobHookNameAvailableResponse`

Name	Type	Description
`is_available`	`bool`	Whether or not the hook name is available

`IsJobNameAvailableRequest`

Name	Type	Description
`name`	`string`	The name to check for availability
`account_id`	`string`	The unique identifier of the account to check for availability

`IsJobNameAvailableResponse`

Name	Type	Description
`is_available`	`bool`	Whether the name is available

`Job`

Name	Type	Description
`id`	`string`	The unique identifier of the job
`created_by_user_id`	`string`
`created_at`	`google.protobuf.Timestamp`
`updated_by_user_id`	`string`
`updated_at`	`google.protobuf.Timestamp`
`name`	`string`	The unique, friendly name of the job
`source`	`JobSource`	The source connection configuration
`destinations`	repeated `JobDestination`	The list of destinations that will be used to store the data
`mappings`	repeated `JobMapping`	The list of mappings that will be used to transform the data
`cron_schedule`	optional `string`	The cron schedule that will be used to trigger the job
`account_id`	`string`	The account identifier that a job is associated with
`sync_options`	`ActivityOptions`	Specify timeout and retry options for data synchronization activities Data sync activities are any piece of work that involves actually synchronizing data from a source to a destination For the data sync and generate jobs, this will be applied per table
`workflow_options`	`WorkflowOptions`	Specify timeouts and other workflow options for the underlying temporal workflow
`virtual_foreign_keys`	repeated `VirtualForeignConstraint`	Any virtual foreign keys that are configured as a part of this job
`job_type`	`JobTypeConfig`	The type of job

`JobDestination`

Name	Type	Description
`connection_id`	`string`	The connection id to use for the job destination
`options`	`JobDestinationOptions`	The destination options to use for the job destination
`id`	`string`	The unique identifier of the job destination

`JobDestinationOptions`

Name	Type	Description
`postgres_options`	`PostgresDestinationConnectionOptions`
`aws_s3_options`	`AwsS3DestinationConnectionOptions`
`mysql_options`	`MysqlDestinationConnectionOptions`
`mongodb_options`	`MongoDBDestinationConnectionOptions`
`gcp_cloudstorage_options`	`GcpCloudStorageDestinationConnectionOptions`	Destination Connecton options for Google Cloud Storage
`dynamodb_options`	`DynamoDBDestinationConnectionOptions`	Destination Connection options for DynamoDB
`mssql_options`	`MssqlDestinationConnectionOptions`	Destination Connection options for Microsoft SQL Server

`JobHook`

Name	Type	Description
`id`	`string`	The unique identifier of this hook.
`name`	`string`	Name of the hook for display/reference.
`description`	`string`	Description of what this hook does.
`job_id`	`string`	The unique identifier of the job this hook belongs to.
`config`	`JobHookConfig`	Hook-type specific configuration.
`created_by_user_id`	`string`	The user that created this hook.
`created_at`	`google.protobuf.Timestamp`	The time this hook was created.
`updated_by_user_id`	`string`	The user that last updated this hook.
`updated_at`	`google.protobuf.Timestamp`	The last time this hook was updated.
`enabled`	`bool`	Whether or not the hook is enabled.
`priority`	`uint32`	The priority of the hook (0-100). This determines the execution order. Lower values are higher priority (priority=0 is the highest). Tie Breaking is determined by the following: (priority, created_at, id) in ascending order.

`JobHookConfig`

Contains the specific hook type configurations.

Name	Type	Description
`sql`	`JobHookConfig.JobSqlHook`	Configuration for SQL-specific hooks.

`JobHookConfig.JobSqlHook`

Configuration for SQL-based hooks

Name	Type	Description
`query`	`string`	The SQL query to execute
`connection_id`	`string`	Unique identifier of the Vydon connection to run this hook for. Must be a connection id that is present in the job.
`timing`	`JobHookConfig.JobSqlHook.Timing`	The timing of when the hook will run

`JobHookConfig.JobSqlHook.Timing`

Name	Type	Description
`pre_sync`	`JobHookTimingPreSync`	A Pre-Sync timing. Will run before the first table sync. Will run before Truncation, if enabled. Will run before Schema Init, if enabled.
`post_sync`	`JobHookTimingPostSync`	A Post-Sync timing. Will run after the last table sync.

`JobHookTimingPostSync`

Configures the job hook to run after the last table sync.

Name	Type	Description

`JobHookTimingPreSync`

Configures the job hook to run before the first table sync. Will run before Truncation, if enabled. Will run before Schema Init, if enabled.

Name	Type	Description

`JobMapping`

Name	Type	Description
`schema`	`string`	The database schema
`table`	`string`	The database table.
`column`	`string`	The column in the configured schema.table
`transformer`	`JobMappingTransformer`	The transformer configuration that will be applied to each cell in the column

`JobMappingTransformer`

Name	Type	Description
`config`	`TransformerConfig`	The transformer configuration

`JobNextRuns`

Name	Type	Description
`next_run_times`	repeated `google.protobuf.Timestamp`	The list of next run times

`JobRecentRun`

Name	Type	Description
`start_time`	`google.protobuf.Timestamp`	The start time of the job run
`job_run_id`	`string`	The unique identifier of the job run

`JobRun`

Name	Type	Description
`id`	`string`	The id of the job run. This will currently be equivalent to the temporal workflow id
`job_id`	`string`	The unique identifier of the job id this run is associated with
`name`	`string`	The name of the job run.
`status`	`JobRunStatus`	the status of the job run
`started_at`	`google.protobuf.Timestamp`	A timestamp of when the run started
`completed_at`	optional `google.protobuf.Timestamp`	Available if the run completed or has not yet been archived by the system
`pending_activities`	repeated `PendingActivity`	Pending activities are only returned when retrieving a specific job run and will not be returned when requesting job runs in list format

`JobRunEvent`

Name	Type	Description
`id`	`int64`	The unique identifier of the event
`type`	`string`	The type of the event
`start_time`	`google.protobuf.Timestamp`	The start time of the event
`close_time`	`google.protobuf.Timestamp`	The close time of the event
`metadata`	`JobRunEventMetadata`	The metadata of the event
`tasks`	repeated `JobRunEventTask`	The list of tasks associated with the event

`JobRunEventMetadata`

Name	Type	Description
`sync_metadata`	`JobRunSyncMetadata`	The metadata of the sync

`JobRunEventTask`

Name	Type	Description
`id`	`int64`	The unique identifier of the task
`type`	`string`	The type of the task
`event_time`	`google.protobuf.Timestamp`	The time of the event
`error`	`JobRunEventTaskError`	The error of the task

`JobRunEventTaskError`

Name	Type	Description
`message`	`string`	The message of the error
`retry_state`	`string`	The retry state of the error

`JobRunSyncMetadata`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table of the sync

`JobSource`

Name	Type	Description
`options`	`JobSourceOptions`	The source options to use for the job

`JobSourceOptions`

Name	Type	Description
`postgres`	`PostgresSourceConnectionOptions`
`aws_s3`	`AwsS3SourceConnectionOptions`
`mysql`	`MysqlSourceConnectionOptions`
`generate`	`GenerateSourceOptions`
`ai_generate`	`AiGenerateSourceOptions`
`mongodb`	`MongoDBSourceConnectionOptions`
`dynamodb`	`DynamoDBSourceConnectionOptions`
`mssql`	`MssqlSourceConnectionOptions`

`JobSourceSqlSubetSchemas`

Name	Type	Description
`postgres_subset`	`PostgresSourceSchemaSubset`	The list of schemas (and their tables) along with any configuration options that will be used.
`mysql_subset`	`MysqlSourceSchemaSubset`	The list of schemas (and their tables) along with any configuration options that will be used.
`dynamodb_subset`	`DynamoDBSourceSchemaSubset`	The list of tables (and their configuration) that reside within the schema
`mssql_subset`	`MssqlSourceSchemaSubset`	The list of schemas (and their tables) along with any configuration options that will be used.

`JobStatusRecord`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job
`status`	`JobStatus`	The status of the job

`JobTypeConfig`

Specifies the job type along with any specific configuration for that job type

Name	Type	Description
`sync`	`JobTypeConfig.JobTypeSync`	This is the default if no job type is specified
`pii_detect`	`JobTypeConfig.JobTypePiiDetect`

`JobTypeConfig.JobTypePiiDetect`

Name	Type	Description
`data_sampling`	`JobTypeConfig.JobTypePiiDetect.DataSampling`	The configuration for data sampling
`table_scan_filter`	`JobTypeConfig.JobTypePiiDetect.TableScanFilter`	The configuration for filtering tables to scan
`user_prompt`	optional `string`	The user prompt to use for PII detection to help influence the LLM
`incremental`	optional `JobTypeConfig.JobTypePiiDetect.Incremental`	The configuration for incremental PII detection

`JobTypeConfig.JobTypePiiDetect.DataSampling`

The configuration for data sampling

Name	Type	Description
`is_enabled`	`bool`	Whether to enable data sampling

`JobTypeConfig.JobTypePiiDetect.IncludeAll`

Configuration to include all tables

Name	Type	Description

`JobTypeConfig.JobTypePiiDetect.Incremental`

The configuration for incremental PII detection

Name	Type	Description
`is_enabled`	`bool`	Whether to enable incremental PII detection

`JobTypeConfig.JobTypePiiDetect.TableIdentifier`

Identifier for a specific table

Name	Type	Description
`schema`	`string`	The schema name
`table`	`string`	The table name

`JobTypeConfig.JobTypePiiDetect.TablePatterns`

Patterns for matching tables and schemas

Name	Type	Description
`schemas`	repeated `string`	Match entire schemas
`tables`	repeated `JobTypeConfig.JobTypePiiDetect.TableIdentifier`	Match specific tables within schemas

`JobTypeConfig.JobTypePiiDetect.TableScanFilter`

Filter configuration for table scanning

Name	Type	Description
`include_all`	`JobTypeConfig.JobTypePiiDetect.IncludeAll`	Include all tables (default behavior)
`include`	`JobTypeConfig.JobTypePiiDetect.TablePatterns`	Only include specified tables/schemas
`exclude`	`JobTypeConfig.JobTypePiiDetect.TablePatterns`	Include all except specified tables/schemas

`JobTypeConfig.JobTypeSync`

The configuration for a data sync job

Name	Type	Description

`MongoDBDestinationConnectionOptions`

Name	Type	Description

`MongoDBSourceConnectionOptions`

MongoDB connection options for a job source

Name	Type	Description
`connection_id`	`string`	The unique connection id to a mongo connection configuration

`MssqlDestinationConnectionOptions`

Name	Type	Description
`truncate_table`	`MssqlTruncateTableConfig`	Whether to truncate the table before inserting data
`init_table_schema`	`bool`	Whether to initialize the table schema before inserting data
`on_conflict`	`MssqlOnConflictConfig`	The configuration for handling conflicts when inserting data
`skip_foreign_key_violations`	`bool`	Insert all valid records, skipping any that violate foreign key constraints.
`batch`	`BatchConfig`	Configure batching options to handle how much data is sent to your database at once.
`max_in_flight`	optional `uint32`	Determines the maximum number of parallel batched inserts.

`MssqlOnConflictConfig`

Name	Type	Description
`do_nothing`	`bool`	Whether to do nothing when a conflict occurs

`MssqlSourceConnectionOptions`

Name	Type	Description
`halt_on_new_column_addition`	`bool`	Whether to halt the job if a new column is added Deprecated: Use new_column_addition_strategy instead
`schemas`	repeated `MssqlSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used.
`connection_id`	`string`	The unique connection id to a mssql connection configuration
`subset_by_foreign_key_constraints`	`bool`	Whether to subset the table by foreign key constraints
`column_removal_strategy`	`MssqlSourceConnectionOptions.ColumnRemovalStrategy`	Provide a strategy of what to do in the event Vydon encounters a column that is removed from the source table.
`new_column_addition_strategy`	`MssqlSourceConnectionOptions.NewColumnAdditionStrategy`	Provide a strategy of what to do in the event Vydon encounters an unmapped column for the job's mapped tables.

`MssqlSourceConnectionOptions.ColumnRemovalStrategy`

Name	Type	Description
`halt_job`	`MssqlSourceConnectionOptions.ColumnRemovalStrategy.HaltJob`	halt job if a column is removed
`continue_job`	`MssqlSourceConnectionOptions.ColumnRemovalStrategy.ContinueJob`	continue job if a column is removed

`MssqlSourceConnectionOptions.ColumnRemovalStrategy.ContinueJob`

Configuration for the ContinueJob strategy

Name	Type	Description

`MssqlSourceConnectionOptions.ColumnRemovalStrategy.HaltJob`

Configuration for the HaltJob strategy

Name	Type	Description

`MssqlSourceConnectionOptions.NewColumnAdditionStrategy`

Name	Type	Description
`halt_job`	`MssqlSourceConnectionOptions.NewColumnAdditionStrategy.HaltJob`	halt job if a new column is detected.
`passthrough`	`MssqlSourceConnectionOptions.NewColumnAdditionStrategy.Passthrough`	pass the new column through as is.

`MssqlSourceConnectionOptions.NewColumnAdditionStrategy.HaltJob`

Configuration for the HaltJob strategy

Name	Type	Description

`MssqlSourceConnectionOptions.NewColumnAdditionStrategy.Passthrough`

Configuration for the Passthrough strategy

Name	Type	Description

`MssqlSourceSchemaOption`

Name	Type	Description
`schema`	`string`	The database schema
`tables`	repeated `MssqlSourceTableOption`	The list of tables (and their configuration) that reside within the schema

`MssqlSourceSchemaSubset`

Name	Type	Description
`mssql_schemas`	repeated `MssqlSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used.

`MssqlSourceTableOption`

Name	Type	Description
`table`	`string`	The table that will be used to subset the data for.
`where_clause`	optional `string`	This is not a parameterized query and must be valid. Intended to be everything after the WHERE keyword. Is used to subset the table.

`MssqlTruncateTableConfig`

Name	Type	Description
`truncate_before_insert`	`bool`	Whether to truncate the table before inserting data

`MysqlDestinationConnectionOptions`

Name	Type	Description
`truncate_table`	`MysqlTruncateTableConfig`	Whether to truncate the table before inserting data
`init_table_schema`	`bool`	Whether to initialize the table schema before inserting data
`on_conflict`	`MysqlOnConflictConfig`	The configuration for handling conflicts when inserting data
`skip_foreign_key_violations`	`bool`	Insert all valid records, skipping any that violate foreign key constraints.
`batch`	`BatchConfig`	Configure batching options to handle how much data is sent to your database at once.
`max_in_flight`	optional `uint32`	Determines the maximum number of parallel batched inserts.

`MysqlOnConflictConfig`

Name	Type	Description
`do_nothing`	`bool`	@deprecated - Use strategy nothing instead
`nothing`	`MysqlOnConflictConfig.MysqlOnConflictDoNothing`	Do nothing when a conflict occurs
`update`	`MysqlOnConflictConfig.MysqlOnConflictUpdate`	Update columns when a conflict occurs

`MysqlOnConflictConfig.MysqlOnConflictDoNothing`

Do nothing strategy

Name	Type	Description

`MysqlOnConflictConfig.MysqlOnConflictUpdate`

Update strategy // List of table-specific update configurations repeated MysqlOnConflictSchemaUpdate schemas = 1;

Name	Type	Description

`MysqlSourceConnectionOptions`

Name	Type	Description
`halt_on_new_column_addition`	`bool`	Whether to halt the job if a new column is added Deprecated: Use new_column_addition_strategy instead
`schemas`	repeated `MysqlSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used.
`connection_id`	`string`	The unique connection id to a mysql connection configuration
`subset_by_foreign_key_constraints`	`bool`	Whether to subset the table by foreign key constraints
`column_removal_strategy`	`MysqlSourceConnectionOptions.ColumnRemovalStrategy`	Provide a strategy of what to do in the event Vydon encounters a column that is removed from the source table.
`new_column_addition_strategy`	`MysqlSourceConnectionOptions.NewColumnAdditionStrategy`	Provide a strategy of what to do in the event Vydon encounters an unmapped column for the job's mapped tables.

`MysqlSourceConnectionOptions.ColumnRemovalStrategy`

Name	Type	Description
`halt_job`	`MysqlSourceConnectionOptions.ColumnRemovalStrategy.HaltJob`	halt job if a column is removed
`continue_job`	`MysqlSourceConnectionOptions.ColumnRemovalStrategy.ContinueJob`	continue job if a column is removed

`MysqlSourceConnectionOptions.ColumnRemovalStrategy.ContinueJob`

Configuration for the ContinueJob strategy

Name	Type	Description

`MysqlSourceConnectionOptions.ColumnRemovalStrategy.HaltJob`

Configuration for the HaltJob strategy

Name	Type	Description

`MysqlSourceConnectionOptions.NewColumnAdditionStrategy`

Name	Type	Description
`halt_job`	`MysqlSourceConnectionOptions.NewColumnAdditionStrategy.HaltJob`	halt job if a new column is detected.
`auto_map`	`MysqlSourceConnectionOptions.NewColumnAdditionStrategy.AutoMap`	automatically handle unmapped columns. It handles this by using the DBs default/nullable values. If this doesn't exist, will fall back to configuring generators for supported datatypes. If none of the criteria above can be met, the job run will fail to prevent leaking of PII.
`passthrough`	`MysqlSourceConnectionOptions.NewColumnAdditionStrategy.Passthrough`	pass the new column through as is.

`MysqlSourceConnectionOptions.NewColumnAdditionStrategy.AutoMap`

Configuration for the AutoMap strategy

Name	Type	Description

`MysqlSourceConnectionOptions.NewColumnAdditionStrategy.HaltJob`

Configuration for the HaltJob strategy

Name	Type	Description

`MysqlSourceConnectionOptions.NewColumnAdditionStrategy.Passthrough`

Configuration for the Passthrough strategy

Name	Type	Description

`MysqlSourceSchemaOption`

Name	Type	Description
`schema`	`string`	The database schema
`tables`	repeated `MysqlSourceTableOption`	The list of tables (and their configuration) that reside within the schema

`MysqlSourceSchemaSubset`

Name	Type	Description
`mysql_schemas`	repeated `MysqlSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used.

`MysqlSourceTableOption`

Name	Type	Description
`table`	`string`	The table that will be used to subset the data for.
`where_clause`	optional `string`	This is not a parameterized query and must be valid. Intended to be everything after the WHERE keyword. Is used to subset the table.

`MysqlTruncateTableConfig`

Name	Type	Description
`truncate_before_insert`	`bool`	Whether to truncate the table before inserting data

`NewJobHook`

Name	Type	Description
`name`	`string`	Name of the hook for display/reference.
`description`	`string`	Description of what this hook does.
`config`	`JobHookConfig`	Hook-type specific configuration.
`enabled`	`bool`	Whether or not the hook is enabled.
`priority`	`uint32`	The priority of the hook (0-100). This determines the execution order. Lower values are higher priority (priority=0 is the highest). Tie Breaking is determined by the following: (priority, created_at, id) in ascending order.

`PauseJobRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job
`pause`	`bool`	Whether to pause or unpause the job
`note`	optional `string`	An optional note to be associated with the pause

`PauseJobResponse`

Name	Type	Description
`job`	`Job`	The job that was updated

`PendingActivity`

Name	Type	Description
`status`	`ActivityStatus`	The status of the activity
`activity_name`	`string`	The name of the activity
`last_failure`	optional `ActivityFailure`	The last failure of the activity

`PiiDetectionReport`

The pii detection report

Name	Type	Description
`tables`	repeated `PiiDetectionReport.TableReport`	The report for each table where PII was detected. If no PII was detected, the table will not be included.

`PiiDetectionReport.TableReport`

The report for a table

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The name of the table
`columns`	repeated `PiiDetectionReport.TableReport.ColumnReport`	The report for each column

`PiiDetectionReport.TableReport.ColumnReport`

The report for a column

Name	Type	Description
`column`	`string`	The name of the column
`regex_report`	optional `PiiDetectionReport.TableReport.ColumnReport.Regex`	The regex report. Will be empty if no regex matched the column name.
`llm_report`	optional `PiiDetectionReport.TableReport.ColumnReport.LLM`	The LLM report. Will be empty if the LLM determined the column name (and optionally the data) was not PII.

`PiiDetectionReport.TableReport.ColumnReport.LLM`

The LLM report

Name	Type	Description
`category`	`string`	The category of the column
`confidence`	`float`	The confidence of the column from 0.0-1.0

`PiiDetectionReport.TableReport.ColumnReport.Regex`

The regex report

Name	Type	Description
`category`	`string`	The category of the column

`PostgresDestinationConnectionOptions`

Name	Type	Description
`truncate_table`	`PostgresTruncateTableConfig`	Whether to truncate the table before inserting data
`init_table_schema`	`bool`	Whether to initialize the table schema before inserting data
`on_conflict`	`PostgresOnConflictConfig`	The configuration for handling conflicts when inserting data
`skip_foreign_key_violations`	`bool`	Whether to skip records that violate foreign key constraints
`batch`	`BatchConfig`	Configure batching options to handle how much data is sent to your database at once.
`max_in_flight`	optional `uint32`	Determines the maximum number of parallel batched inserts.

`PostgresOnConflictConfig`

Name	Type	Description
`do_nothing`	`bool`	@deprecated - Use strategy nothing instead
`nothing`	`PostgresOnConflictConfig.PostgresOnConflictDoNothing`	Do nothing when a conflict occurs
`update`	`PostgresOnConflictConfig.PostgresOnConflictUpdate`	Update columns when a conflict occurs

`PostgresOnConflictConfig.PostgresOnConflictDoNothing`

Do nothing strategy

Name	Type	Description

`PostgresOnConflictConfig.PostgresOnConflictUpdate`

Update strategy // List of table-specific update configurations repeated PostgresOnConflictSchemaUpdate schemas = 1;

Name	Type	Description

`PostgresSourceConnectionOptions`

Name	Type	Description
`schemas`	repeated `PostgresSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used.
`connection_id`	`string`	The unique connection id to a postgres connection configuration
`subset_by_foreign_key_constraints`	`bool`	Whether to subset the table by foreign key constraints
`new_column_addition_strategy`	`PostgresSourceConnectionOptions.NewColumnAdditionStrategy`	Provide a strategy of what to do in the event Vydon encounters an unmapped column for the job's mapped tables.
`column_removal_strategy`	`PostgresSourceConnectionOptions.ColumnRemovalStrategy`	Provide a strategy of what to do in the event Vydon encounters a column that is removed from the source table.

`PostgresSourceConnectionOptions.ColumnRemovalStrategy`

Name	Type	Description
`halt_job`	`PostgresSourceConnectionOptions.ColumnRemovalStrategy.HaltJob`	halt job if a column is removed
`continue_job`	`PostgresSourceConnectionOptions.ColumnRemovalStrategy.ContinueJob`	continue job if a column is removed

`PostgresSourceConnectionOptions.ColumnRemovalStrategy.ContinueJob`

Configuration for the ContinueJob strategy

Name	Type	Description

`PostgresSourceConnectionOptions.ColumnRemovalStrategy.HaltJob`

Configuration for the HaltJob strategy

Name	Type	Description

`PostgresSourceConnectionOptions.NewColumnAdditionStrategy`

Name	Type	Description
`halt_job`	`PostgresSourceConnectionOptions.NewColumnAdditionStrategy.HaltJob`	halt job if a new column is detected.
`auto_map`	`PostgresSourceConnectionOptions.NewColumnAdditionStrategy.AutoMap`	automatically handle unmapped columns. It handles this by using the DBs default/nullable values. If this doesn't exist, will fall back to configuring generators for supported datatypes. If none of the criteria above can be met, the job run will fail to prevent leaking of PII.
`passthrough`	`PostgresSourceConnectionOptions.NewColumnAdditionStrategy.Passthrough`	pass the new column through as is.

`PostgresSourceConnectionOptions.NewColumnAdditionStrategy.AutoMap`

Configuration for the AutoMap strategy

Name	Type	Description

`PostgresSourceConnectionOptions.NewColumnAdditionStrategy.HaltJob`

Configuration for the HaltJob strategy

Name	Type	Description

`PostgresSourceConnectionOptions.NewColumnAdditionStrategy.Passthrough`

Configuration for the Passthrough strategy

Name	Type	Description

`PostgresSourceSchemaOption`

Name	Type	Description
`schema`	`string`	The database schema
`tables`	repeated `PostgresSourceTableOption`	The list of tables (and their configuration) that reside within the schema

`PostgresSourceSchemaSubset`

Name	Type	Description
`postgres_schemas`	repeated `PostgresSourceSchemaOption`	The list of schemas (and their tables) along with any configuration options that will be used.

`PostgresSourceTableOption`

Name	Type	Description
`table`	`string`	The table that will be used subset the data for.
`where_clause`	optional `string`	This is not a parameterized query and must be valid. Intended to be everything after the WHERE keyword. Is used to subset the table.

`PostgresTruncateTableConfig`

Name	Type	Description
`truncate_before_insert`	`bool`	Whether to truncate the table before inserting data
`cascade`	`bool`	Whether to cascade the truncate to child tables

`RetryPolicy`

Defines the retry policy for an activity

Name	Type	Description
`maximum_attempts`	optional `int32`	Maximum number of attempts. When exceeded the retries stop even if not expired yet. If not set or set to 0, it means unlimited, and rely on activity ScheduleToCloseTimeout to stop.

`RunContextKey`

Name	Type	Description
`job_run_id`	`string`	The Vydon Run ID
`external_id`	`string`	An opaque identifier that will be used to store specific items
`account_id`	`string`	The Vydon Account ID

`SetJobHookEnabledRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the hook
`enabled`	`bool`	Whether or not the hook is enabled.

`SetJobHookEnabledResponse`

Name	Type	Description
`hook`	`JobHook`	The updated job hook

`SetJobSourceSqlConnectionSubsetsRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job to update subsets for
`schemas`	`JobSourceSqlSubetSchemas`	The subset configuration
`subset_by_foreign_key_constraints`	`bool`	Whether or not to have subsets follow foreign key constraints (for connections that support it)

`SetJobSourceSqlConnectionSubsetsResponse`

Name	Type	Description
`job`	`Job`	The job that was updated

`SetJobSyncOptionsRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job
`sync_options`	`ActivityOptions`	The sync options object. The entire object must be provided and will fully overwrite the previous result

`SetJobSyncOptionsResponse`

Name	Type	Description
`job`	`Job`	The updated job

`SetJobWorkflowOptionsRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job
`worfklow_options`	`WorkflowOptions`	The workflow options object. The entire object must be provided and will fully overwrite the previous result

`SetJobWorkflowOptionsResponse`

Name	Type	Description
`job`	`Job`	The updated job

`SetRunContextRequest`

Name	Type	Description
`id`	`RunContextKey`	The run context key
`value`	`bytes`	An opaque value that is to be determined by the key

`SetRunContextResponse`

Name	Type	Description

`SetRunContextsRequest`

Name	Type	Description
`id`	`RunContextKey`	The run context key
`value`	`bytes`	An opaque value that is to be determined by the key

`SetRunContextsResponse`

Name	Type	Description

`TableError`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table of the error
`error_reports`	repeated `TableError.TableErrorReport`	The list of error reports

`TableError.TableErrorReport`

Database error report

Name	Type	Description
`code`	`TableError.TableErrorCode`	The error code
`message`	`string`	The error message

`TerminateJobRunRequest`

Name	Type	Description
`job_run_id`	`string`	The unique identifier of the job run
`account_id`	`string`	The unique identifier of the account

`TerminateJobRunResponse`

Name	Type	Description

`UpdateJobDestinationConnectionRequest`

Name	Type	Description
`job_id`	`string`	The unique identifier of the job
`connection_id`	`string`	The unique identifier of the connection
`options`	`JobDestinationOptions`	The destination connection options
`destination_id`	`string`	The unique identifier of the destination

`UpdateJobDestinationConnectionResponse`

Name	Type	Description
`job`	`Job`	The job that was updated

`UpdateJobHookRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the hook
`name`	`string`	Name of the hook for display/reference.
`description`	`string`	Description of what this hook does.
`config`	`JobHookConfig`	Hook-type specific configuration.
`enabled`	`bool`	Whether or not the hook is enabled.
`priority`	`uint32`	The priority of the hook (0-100). This determines the execution order. Lower values are higher priority (priority=0 is the highest). Tie Breaking is determined by the following: (priority, created_at, id) in ascending order.

`UpdateJobHookResponse`

Name	Type	Description
`hook`	`JobHook`	The updated job hook

`UpdateJobScheduleRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job
`cron_schedule`	optional `string`	The new cron schedule

`UpdateJobScheduleResponse`

Name	Type	Description
`job`	`Job`	The job that was updated

`UpdateJobSourceConnectionRequest`

Name	Type	Description
`id`	`string`	The unique identifier of the job
`source`	`JobSource`	The new source connection configuration
`mappings`	repeated `JobMapping`	The new list of mappings that will be used to transform the data
`virtual_foreign_keys`	repeated `VirtualForeignConstraint`	The new list of virtual foreign keys that will be used to further constrain the data ontop of the database defined constraints
`job_type`	`JobTypeConfig`	The new job type configuration

`UpdateJobSourceConnectionResponse`

Name	Type	Description
`job`	`Job`

`ValidateJobMappingsRequest`

Name	Type	Description
`account_id`	`string`	The unique identifier of the account
`mappings`	repeated `JobMapping`	The list of mappings to validate
`connection_id`	`string`	The unique identifier of the connection
`virtual_foreign_keys`	repeated `VirtualForeignConstraint`	The list of virtual foreign keys
`job_source`	optional `JobSource`	The source options of the job

`ValidateJobMappingsResponse`

Name	Type	Description
`column_errors`	repeated `ColumnError`	The list of column errors
`database_errors`	`DatabaseError`	The list of database errors
`column_warnings`	repeated `ColumnWarning`	The list of column warnings
`table_errors`	repeated `TableError`	The list of table errors

`ValidateSchemaRequest`

Name	Type	Description
`mappings`	repeated `JobMapping`	The list of mappings to validate
`connection_id`	`string`	The unique identifier of the connection to validate

`ValidateSchemaResponse`

Name	Type	Description
`missing_columns`	repeated `DatabaseColumn`	The list columns missing in destination
`extra_columns`	repeated `DatabaseColumn`	The list of extra columns in destination and not in source
`missing_tables`	repeated `ValidateSchemaResponse.Table`	The list of tables missing in destination
`missing_schemas`	repeated `string`	The list of schemas missing in destination

`ValidateSchemaResponse.Table`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table

`VirtualForeignConstraint`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table of the virtual foreign key
`columns`	repeated `string`	The list of columns
`foreign_key`	`VirtualForeignKey`	The foreign key

`VirtualForeignKey`

Name	Type	Description
`schema`	`string`	The schema of the table
`table`	`string`	The table of the virtual foreign key
`columns`	repeated `string`	The list of columns

`WorkflowOptions`

Config that contains various timeouts that are configured in the underlying temporal workflow More options will come in the future as needed

Name	Type	Description
`run_timeout`	optional `int64`	The timeout for a single workflow run. Measured in seconds

Enums

`ActivityStatus`

Name	Number	Description
`ACTIVITY_STATUS_UNSPECIFIED`	`0`
`ACTIVITY_STATUS_SCHEDULED`	`1`
`ACTIVITY_STATUS_STARTED`	`2`
`ACTIVITY_STATUS_CANCELED`	`3`
`ACTIVITY_STATUS_FAILED`	`4`

`AwsS3DestinationConnectionOptions.StorageClass`

Name	Number	Description
`STORAGE_CLASS_UNSPECIFIED`	`0`
`STORAGE_CLASS_STANDARD`	`1`
`STORAGE_CLASS_REDUCED_REDUNDANCY`	`2`
`STORAGE_CLASS_GLACIER`	`3`
`STORAGE_CLASS_STANDARD_IA`	`4`
`STORAGE_CLASS_ONEZONE_IA`	`5`
`STORAGE_CLASS_INTELLIGENT_TIERING`	`6`
`STORAGE_CLASS_DEEP_ARCHIVE`	`7`

`ColumnError.ColumnErrorCode`

An enumeration of column error codes

Name	Number	Description
`COLUMN_ERROR_CODE_UNSPECIFIED`	`0`	Default unspecified value
`COLUMN_ERROR_CODE_NOT_FOUND_IN_SOURCE`	`1`	Column not found in source database
`COLUMN_ERROR_CODE_NOT_FOUND_IN_MAPPING`	`2`	Column not found in job mapping
`COLUMN_ERROR_CODE_REQUIRED_COLUMN_NOT_FOUND_IN_MAPPING`	`3`	Required column not found in job mapping
`COLUMN_ERROR_CODE_REQUIRED_FOREIGN_KEY_NOT_FOUND_IN_MAPPING`	`4`	Required foreign key not found in job mapping
`COLUMN_ERROR_CODE_UNSUPPORTED_CIRCULAR_DEPENDENCY_AT_LEAST_ONE_NULLABLE`	`5`	Unsupported circular dependency detected
`COLUMN_ERROR_CODE_VFK_SOURCE_COLUMN_NOT_FOUND_IN_MAPPING`	`6`	Virtual foreign key source column not found in mapping
`COLUMN_ERROR_CODE_VFK_SOURCE_COLUMN_NOT_FOUND_IN_SOURCE`	`7`	Virtual foreign key source column not found in source
`COLUMN_ERROR_CODE_VFK_TARGET_COLUMN_NOT_FOUND_IN_MAPPING`	`8`	Virtual foreign key target column not found in mapping
`COLUMN_ERROR_CODE_VFK_TARGET_COLUMN_NOT_FOUND_IN_SOURCE`	`9`	Virtual foreign key target column not found in source
`COLUMN_ERROR_CODE_VFK_COLUMN_DATATYPE_MISMATCH`	`10`	Virtual foreign key column datatype mismatch
`COLUMN_ERROR_CODE_VFK_SOURCE_COLUMN_NOT_UNIQUE`	`11`	Virtual foreign key source column not unique

`ColumnWarning.ColumnWarningCode`

An enumeration of column warning codes

Name	Number	Description
`COLUMN_WARNING_CODE_UNSPECIFIED`	`0`	Default unspecified value
`COLUMN_WARNING_CODE_NOT_FOUND_IN_SOURCE`	`1`	Column not found in source database
`COLUMN_WARNING_CODE_NOT_FOUND_IN_MAPPING`	`2`	Column not found in job mapping

`DatabaseError.DatabaseErrorCode`

An enumeration of database error codes

Name	Number	Description
`DATABASE_ERROR_CODE_UNSPECIFIED`	`0`	Default unspecified value
`DATABASE_ERROR_CODE_UNSUPPORTED_CIRCULAR_DEPENDENCY_AT_LEAST_ONE_NULLABLE`	`1`	Unsupported circular dependency detected
`DATABASE_ERROR_CODE_VFK_COLUMN_MISMATCH`	`2`	Virtual foreign key column mismatch

`GetActiveJobHooksByTimingRequest.Timing`

Name	Number	Description
`TIMING_UNSPECIFIED`	`0`	If unspecified, returns all active job hooks
`TIMING_PRESYNC`	`1`	Only returns presync hooks
`TIMING_POSTSYNC`	`2`	Only returns postsync hooks

`JobRunStatus`

An enumeration of job run statuses.

Name	Number	Description
`JOB_RUN_STATUS_UNSPECIFIED`	`0`	if the job run status is unknown
`JOB_RUN_STATUS_PENDING`	`1`	the run is pending and has not started yet
`JOB_RUN_STATUS_RUNNING`	`2`	the run is currently in progress
`JOB_RUN_STATUS_COMPLETE`	`3`	the run has successfully completed
`JOB_RUN_STATUS_ERROR`	`4`	the run ended with an error
`JOB_RUN_STATUS_CANCELED`	`5`	the run was cancelled
`JOB_RUN_STATUS_TERMINATED`	`6`	the run was terminated
`JOB_RUN_STATUS_FAILED`	`7`	the run ended in failure
`JOB_RUN_STATUS_TIMED_OUT`	`8`	the run was ended pre-maturely due to timeout

`JobStatus`

Name	Number	Description
`JOB_STATUS_UNSPECIFIED`	`0`
`JOB_STATUS_ENABLED`	`1`
`JOB_STATUS_PAUSED`	`3`
`JOB_STATUS_DISABLED`	`4`

`LogLevel`

Name	Number	Description
`LOG_LEVEL_UNSPECIFIED`	`0`
`LOG_LEVEL_DEBUG`	`1`
`LOG_LEVEL_INFO`	`2`
`LOG_LEVEL_WARN`	`3`
`LOG_LEVEL_ERROR`	`4`

`LogWindow`

Name	Number	Description
`LOG_WINDOW_NO_TIME_UNSPECIFIED`	`0`
`LOG_WINDOW_FIFTEEN_MIN`	`1`
`LOG_WINDOW_ONE_HOUR`	`2`
`LOG_WINDOW_ONE_DAY`	`3`

`TableError.TableErrorCode`

An enumeration of table error codes

Name	Number	Description
`TABLE_ERROR_CODE_UNSPECIFIED`	`0`	Default unspecified value
`TABLE_ERROR_CODE_TABLE_NOT_FOUND_IN_SOURCE`	`1`	Table not found in source database
`TABLE_ERROR_CODE_VFK_SOURCE_TABLE_NOT_FOUND_IN_MAPPING`	`2`	Virtual foreign key source table not found in mapping
`TABLE_ERROR_CODE_VFK_SOURCE_TABLE_NOT_FOUND_IN_SOURCE`	`3`	Virtual foreign key source table not found in source
`TABLE_ERROR_CODE_VFK_TARGET_TABLE_NOT_FOUND_IN_MAPPING`	`4`	Virtual foreign key target table not found in mapping
`TABLE_ERROR_CODE_VFK_TARGET_TABLE_NOT_FOUND_IN_SOURCE`	`5`	Virtual foreign key target table not found in source

Services

`JobService`

Service that handles jobs, runs, and hooks

`GetJobs`

Method	`GetJobs`
Request	`GetJobsRequest`
Response	`GetJobsResponse`
Description	Returns a list of jobs by either account or job

`GetJob`

Method	`GetJob`
Request	`GetJobRequest`
Response	`GetJobResponse`
Description	Returns a specific job

`CreateJob`

Method	`CreateJob`
Request	`CreateJobRequest`
Response	`CreateJobResponse`
Description	Creates a new job

`DeleteJob`

Method	`DeleteJob`
Request	`DeleteJobRequest`
Response	`DeleteJobResponse`
Description	Deletes a job

`IsJobNameAvailable`

Method	`IsJobNameAvailable`
Request	`IsJobNameAvailableRequest`
Response	`IsJobNameAvailableResponse`
Description	Checks if a job name is available

`UpdateJobSchedule`

Method	`UpdateJobSchedule`
Request	`UpdateJobScheduleRequest`
Response	`UpdateJobScheduleResponse`
Description	Updates the schedule of a job

`UpdateJobSourceConnection`

Method	`UpdateJobSourceConnection`
Request	`UpdateJobSourceConnectionRequest`
Response	`UpdateJobSourceConnectionResponse`
Description	Updates the source connection of a job

`SetJobSourceSqlConnectionSubsets`

Method	`SetJobSourceSqlConnectionSubsets`
Request	`SetJobSourceSqlConnectionSubsetsRequest`
Response	`SetJobSourceSqlConnectionSubsetsResponse`
Description	Sets the source sql connection subsets of a job

`UpdateJobDestinationConnection`

Method	`UpdateJobDestinationConnection`
Request	`UpdateJobDestinationConnectionRequest`
Response	`UpdateJobDestinationConnectionResponse`
Description	Updates the destination connection of a job

`DeleteJobDestinationConnection`

Method	`DeleteJobDestinationConnection`
Request	`DeleteJobDestinationConnectionRequest`
Response	`DeleteJobDestinationConnectionResponse`
Description	Deletes the destination connection of a job

`CreateJobDestinationConnections`

Method	`CreateJobDestinationConnections`
Request	`CreateJobDestinationConnectionsRequest`
Response	`CreateJobDestinationConnectionsResponse`
Description	Creates the destination connections of a job

`PauseJob`

Method	`PauseJob`
Request	`PauseJobRequest`
Response	`PauseJobResponse`
Description	Pauses or unpauses a job

`GetJobRecentRuns`

Method	`GetJobRecentRuns`
Request	`GetJobRecentRunsRequest`
Response	`GetJobRecentRunsResponse`
Description	Returns a list of recently invoked job runs based on the Temporal cron scheduler. This will return a list of job runs that include archived runs

`GetJobNextRuns`

Method	`GetJobNextRuns`
Request	`GetJobNextRunsRequest`
Response	`GetJobNextRunsResponse`
Description	Returns a list of runs that are scheduled for execution based on the Temporal cron scheduler.

`GetJobStatus`

Method	`GetJobStatus`
Request	`GetJobStatusRequest`
Response	`GetJobStatusResponse`
Description	Returns the status of a job

`GetJobStatuses`

Method	`GetJobStatuses`
Request	`GetJobStatusesRequest`
Response	`GetJobStatusesResponse`
Description	Returns the statuses of jobs within an account

`GetJobRuns`

Method	`GetJobRuns`
Request	`GetJobRunsRequest`
Response	`GetJobRunsResponse`
Description	Returns a list of job runs by either account or job

`GetJobRunEvents`

Method	`GetJobRunEvents`
Request	`GetJobRunEventsRequest`
Response	`GetJobRunEventsResponse`
Description	Returns a list of events for a job run to understand more details of the run itself

`GetJobRun`

Method	`GetJobRun`
Request	`GetJobRunRequest`
Response	`GetJobRunResponse`
Description	Returns a specific job run, along with any of its pending activities

`DeleteJobRun`

Method	`DeleteJobRun`
Request	`DeleteJobRunRequest`
Response	`DeleteJobRunResponse`
Description	Deletes a job run

`CreateJobRun`

Method	`CreateJobRun`
Request	`CreateJobRunRequest`
Response	`CreateJobRunResponse`
Description	Creates a new job run

`CancelJobRun`

Method	`CancelJobRun`
Request	`CancelJobRunRequest`
Response	`CancelJobRunResponse`
Description	Cancels a job run. This is a graceful termination and allows the workflow to clean up and exit gracefully.

`TerminateJobRun`

Method	`TerminateJobRun`
Request	`TerminateJobRunRequest`
Response	`TerminateJobRunResponse`
Description	Terminates a job run. This is an immediate termination and will not allow the workflow to clean up and exit gracefully.

`GetJobRunLogsStream`

Method	`GetJobRunLogsStream`
Request	`GetJobRunLogsStreamRequest`
Response	`GetJobRunLogsStreamResponse` stream
Description	Returns a stream of logs from the worker nodes that pertain to a specific job run

`GetJobRunLogs`

Method	`GetJobRunLogs`
Request	`GetJobRunLogsRequest`
Response	`GetJobRunLogsResponse`
Description	Returns a list of logs from the worker nodes that pertain to a specific job run. Equivalent to the stream endpoint, but runs in a unary fashion.

`SetJobWorkflowOptions`

Method	`SetJobWorkflowOptions`
Request	`SetJobWorkflowOptionsRequest`
Response	`SetJobWorkflowOptionsResponse`
Description	Set any job workflow options. Must provide entire object as is it will fully override the previous configuration

`SetJobSyncOptions`

Method	`SetJobSyncOptions`
Request	`SetJobSyncOptionsRequest`
Response	`SetJobSyncOptionsResponse`
Description	Set the job sync options. Must provide entire object as it will fully override the previous configuration

`ValidateJobMappings`

Method	`ValidateJobMappings`
Request	`ValidateJobMappingsRequest`
Response	`ValidateJobMappingsResponse`
Description	Validates that the jobmapping configured can run with table constraints

`ValidateSchema`

Method	`ValidateSchema`
Request	`ValidateSchemaRequest`
Response	`ValidateSchemaResponse`
Description	Validates that the schema is compatible with the job mappings

`GetRunContext`

Method	`GetRunContext`
Request	`GetRunContextRequest`
Response	`GetRunContextResponse`
Description	Gets a run context to be used by a workflow run

`SetRunContext`

Method	`SetRunContext`
Request	`SetRunContextRequest`
Response	`SetRunContextResponse`
Description	Sets a run context to be used by a workflow run

`SetRunContexts`

Method	`SetRunContexts`
Request	`SetRunContextsRequest` stream
Response	`SetRunContextsResponse`
Description	Sets a stream of run contexts to be used by a workflow run

`GetJobHooks`

Method	`GetJobHooks`
Request	`GetJobHooksRequest`
Response	`GetJobHooksResponse`
Description	Retrieves all job hooks

`GetJobHook`

Method	`GetJobHook`
Request	`GetJobHookRequest`
Response	`GetJobHookResponse`
Description	Retrieves a specific job hook

`CreateJobHook`

Method	`CreateJobHook`
Request	`CreateJobHookRequest`
Response	`CreateJobHookResponse`
Description	Creates a new job hook

`DeleteJobHook`

Method	`DeleteJobHook`
Request	`DeleteJobHookRequest`
Response	`DeleteJobHookResponse`
Description	Removes a job hook

`IsJobHookNameAvailable`

Method	`IsJobHookNameAvailable`
Request	`IsJobHookNameAvailableRequest`
Response	`IsJobHookNameAvailableResponse`
Description	Check if a specific job hook name is available

`UpdateJobHook`

Method	`UpdateJobHook`
Request	`UpdateJobHookRequest`
Response	`UpdateJobHookResponse`
Description	Updates a job hook

`SetJobHookEnabled`

Method	`SetJobHookEnabled`
Request	`SetJobHookEnabledRequest`
Response	`SetJobHookEnabledResponse`
Description	Enables or disables a job hook

`GetActiveJobHooksByTiming`

Method	`GetActiveJobHooksByTiming`
Request	`GetActiveJobHooksByTimingRequest`
Response	`GetActiveJobHooksByTimingResponse`
Description	Returns job hooks that are enabled by a specific timing. They will be sorted by priority, created_at, and id ascending.

`GetPiiDetectionReport`

Method	`GetPiiDetectionReport`
Request	`GetPiiDetectionReportRequest`
Response	`GetPiiDetectionReportResponse`
Description

Messages
Enums
Services
- JobService

Messages​

ActivityFailure​

ActivityOptions​

AiGenerateSourceOptions​

AiGenerateSourceSchemaOption​

AiGenerateSourceTableOption​

AwsS3DestinationConnectionOptions​

AwsS3SourceConnectionOptions​

BatchConfig​

CancelJobRunRequest​

CancelJobRunResponse​

ColumnError​

ColumnError.ColumnErrorReport​

ColumnWarning​

ColumnWarning.ColumnWarningReport​

CreateJobDestination​

CreateJobDestinationConnectionsRequest​

CreateJobDestinationConnectionsResponse​

CreateJobHookRequest​

CreateJobHookResponse​

CreateJobRequest​

CreateJobResponse​

CreateJobRunRequest​

CreateJobRunResponse​

DatabaseError​

DatabaseError.DatabaseErrorReport​

DeleteJobDestinationConnectionRequest​

DeleteJobDestinationConnectionResponse​

DeleteJobHookRequest​

DeleteJobHookResponse​

DeleteJobRequest​

DeleteJobResponse​

DeleteJobRunRequest​

DeleteJobRunResponse​

DynamoDBDestinationConnectionOptions​

DynamoDBDestinationTableMapping​

DynamoDBSourceConnectionOptions​

DynamoDBSourceSchemaSubset​

DynamoDBSourceTableOption​

DynamoDBSourceUnmappedTransformConfig​

GcpCloudStorageDestinationConnectionOptions​

GenerateSourceOptions​

GenerateSourceSchemaOption​

GenerateSourceTableOption​

GetActiveJobHooksByTimingRequest​

GetActiveJobHooksByTimingResponse​

GetJobHookRequest​

GetJobHookResponse​

GetJobHooksRequest​

GetJobHooksResponse​

GetJobNextRunsRequest​

GetJobNextRunsResponse​

GetJobRecentRunsRequest​

GetJobRecentRunsResponse​

GetJobRequest​

GetJobResponse​

GetJobRunEventsRequest​

GetJobRunEventsResponse​

GetJobRunLogsRequest​

GetJobRunLogsResponse​

GetJobRunLogsResponse.LogLine​

GetJobRunLogsResponse.LogLine.LabelsEntry​

GetJobRunLogsStreamRequest​

GetJobRunLogsStreamResponse​

GetJobRunLogsStreamResponse.LabelsEntry​

GetJobRunRequest​

GetJobRunResponse​

GetJobRunsRequest​

GetJobRunsResponse​

GetJobStatusRequest​

GetJobStatusResponse​

GetJobStatusesRequest​

GetJobStatusesResponse​

GetJobsRequest​

GetJobsResponse​

GetPiiDetectionReportRequest​

GetPiiDetectionReportResponse​

GetRunContextRequest​

GetRunContextResponse​

IsJobHookNameAvailableRequest​

Messages

`ActivityFailure`

`ActivityOptions`

`AiGenerateSourceOptions`

`AiGenerateSourceSchemaOption`

`AiGenerateSourceTableOption`

`AwsS3DestinationConnectionOptions`

`AwsS3SourceConnectionOptions`

`BatchConfig`

`CancelJobRunRequest`

`CancelJobRunResponse`

`ColumnError`

`ColumnError.ColumnErrorReport`

`ColumnWarning`

`ColumnWarning.ColumnWarningReport`

`CreateJobDestination`

`CreateJobDestinationConnectionsRequest`

`CreateJobDestinationConnectionsResponse`

`CreateJobHookRequest`

`CreateJobHookResponse`

`CreateJobRequest`

`CreateJobResponse`

`CreateJobRunRequest`

`CreateJobRunResponse`

`DatabaseError`

`DatabaseError.DatabaseErrorReport`

`DeleteJobDestinationConnectionRequest`

`DeleteJobDestinationConnectionResponse`

`DeleteJobHookRequest`

`DeleteJobHookResponse`

`DeleteJobRequest`

`DeleteJobResponse`

`DeleteJobRunRequest`

`DeleteJobRunResponse`

`DynamoDBDestinationConnectionOptions`

`DynamoDBDestinationTableMapping`

`DynamoDBSourceConnectionOptions`

`DynamoDBSourceSchemaSubset`

`DynamoDBSourceTableOption`

`DynamoDBSourceUnmappedTransformConfig`

`GcpCloudStorageDestinationConnectionOptions`

`GenerateSourceOptions`

`GenerateSourceSchemaOption`

`GenerateSourceTableOption`

`GetActiveJobHooksByTimingRequest`

`GetActiveJobHooksByTimingResponse`

`GetJobHookRequest`

`GetJobHookResponse`

`GetJobHooksRequest`

`GetJobHooksResponse`

`GetJobNextRunsRequest`

`GetJobNextRunsResponse`

`GetJobRecentRunsRequest`

`GetJobRecentRunsResponse`

`GetJobRequest`

`GetJobResponse`

`GetJobRunEventsRequest`

`GetJobRunEventsResponse`

`GetJobRunLogsRequest`

`GetJobRunLogsResponse`

`GetJobRunLogsResponse.LogLine`

`GetJobRunLogsResponse.LogLine.LabelsEntry`

`GetJobRunLogsStreamRequest`

`GetJobRunLogsStreamResponse`

`GetJobRunLogsStreamResponse.LabelsEntry`

`GetJobRunRequest`

`GetJobRunResponse`

`GetJobRunsRequest`

`GetJobRunsResponse`

`GetJobStatusRequest`

`GetJobStatusResponse`

`GetJobStatusesRequest`

`GetJobStatusesResponse`

`GetJobsRequest`

`GetJobsResponse`

`GetPiiDetectionReportRequest`

`GetPiiDetectionReportResponse`

`GetRunContextRequest`

`GetRunContextResponse`

`IsJobHookNameAvailableRequest`