{ "description": "HyperParameterTuningJob is the Schema for the HyperParameterTuningJobs API", "properties": { "apiVersion": { "description": "APIVersion defines the versioned schema of this representation of an object.\nServers should convert recognized schemas to the latest internal value, and\nmay reject unrecognized values.\nMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#resources", "type": "string" }, "kind": { "description": "Kind is a string value representing the REST resource this object represents.\nServers may infer this from the endpoint the client submits requests to.\nCannot be updated.\nIn CamelCase.\nMore info: https://git.k8s.io/community/contributors/devel/sig-architecture/api-conventions.md#types-kinds", "type": "string" }, "metadata": { "type": "object" }, "spec": { "description": "HyperParameterTuningJobSpec defines the desired state of HyperParameterTuningJob.", "properties": { "autotune": { "description": "Configures SageMaker Automatic model tuning (AMT) to automatically find optimal\nparameters for the following fields:\n\n - ParameterRanges (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobConfig.html#sagemaker-Type-HyperParameterTuningJobConfig-ParameterRanges):\n The names and ranges of parameters that a hyperparameter tuning job can\n optimize.\n\n - ResourceLimits (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ResourceLimits.html):\n The maximum resources that can be used for a training job. These resources\n include the maximum number of training jobs, the maximum runtime of a\n tuning job, and the maximum number of training jobs to run at the same\n time.\n\n - TrainingJobEarlyStoppingType (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobConfig.html#sagemaker-Type-HyperParameterTuningJobConfig-TrainingJobEarlyStoppingType):\n A flag that specifies whether or not to use early stopping for training\n jobs launched by a hyperparameter tuning job.\n\n - RetryStrategy (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTrainingJobDefinition.html#sagemaker-Type-HyperParameterTrainingJobDefinition-RetryStrategy):\n The number of times to retry a training job.\n\n - Strategy (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobConfig.html):\n Specifies how hyperparameter tuning chooses the combinations of hyperparameter\n values to use for the training jobs that it launches.\n\n - ConvergenceDetected (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_ConvergenceDetected.html):\n A flag to indicate that Automatic model tuning (AMT) has detected model\n convergence.", "properties": { "mode": { "type": "string" } }, "type": "object", "additionalProperties": false }, "hyperParameterTuningJobConfig": { "description": "The HyperParameterTuningJobConfig (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobConfig.html)\nobject that describes the tuning job, including the search strategy, the\nobjective metric used to evaluate training jobs, ranges of parameters to\nsearch, and resource limits for the tuning job. For more information, see\nHow Hyperparameter Tuning Works (https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-how-it-works.html).", "properties": { "hyperParameterTuningJobObjective": { "description": "Defines the objective metric for a hyperparameter tuning job. Hyperparameter\ntuning uses the value of this metric to evaluate the training jobs it launches,\nand returns the training job that results in either the highest or lowest\nvalue for this metric, depending on the value you specify for the Type parameter.\nIf you want to define a custom objective metric, see Define metrics and environment\nvariables (https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-metrics-variables.html).", "properties": { "metricName": { "type": "string" }, "type_": { "type": "string" } }, "type": "object", "additionalProperties": false }, "parameterRanges": { "description": "Specifies ranges of integer, continuous, and categorical hyperparameters\nthat a hyperparameter tuning job searches. The hyperparameter tuning job\nlaunches training jobs with hyperparameter values within these ranges to\nfind the combination of values that result in the training job with the best\nperformance as measured by the objective metric of the hyperparameter tuning\njob.\n\nThe maximum number of items specified for Array Members refers to the maximum\nnumber of hyperparameters for each range and also the maximum for the hyperparameter\ntuning job itself. That is, the sum of the number of hyperparameters for\nall the ranges can't exceed the maximum number specified.", "properties": { "autoParameters": { "items": { "description": "The name and an example value of the hyperparameter that you want to use\nin Autotune. If Automatic model tuning (AMT) determines that your hyperparameter\nis eligible for Autotune, an optimal hyperparameter range is selected for\nyou.", "properties": { "name": { "type": "string" }, "valueHint": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "categoricalParameterRanges": { "items": { "description": "A list of categorical hyperparameters to tune.", "properties": { "name": { "type": "string" }, "values": { "items": { "type": "string" }, "type": "array" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "continuousParameterRanges": { "items": { "description": "A list of continuous hyperparameters to tune.", "properties": { "maxValue": { "type": "string" }, "minValue": { "type": "string" }, "name": { "type": "string" }, "scalingType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "integerParameterRanges": { "items": { "description": "For a hyperparameter of the integer type, specifies the range that a hyperparameter\ntuning job searches.", "properties": { "maxValue": { "type": "string" }, "minValue": { "type": "string" }, "name": { "type": "string" }, "scalingType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" } }, "type": "object", "additionalProperties": false }, "resourceLimits": { "description": "Specifies the maximum number of training jobs and parallel training jobs\nthat a hyperparameter tuning job can launch.", "properties": { "maxNumberOfTrainingJobs": { "format": "int64", "type": "integer" }, "maxParallelTrainingJobs": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "strategy": { "description": "The strategy hyperparameter tuning uses to find the best combination of hyperparameters\nfor your model.", "type": "string" }, "trainingJobEarlyStoppingType": { "type": "string" }, "tuningJobCompletionCriteria": { "description": "The job completion criteria.", "properties": { "targetObjectiveMetricValue": { "type": "number" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "hyperParameterTuningJobName": { "description": "The name of the tuning job. This name is the prefix for the names of all\ntraining jobs that this tuning job launches. The name must be unique within\nthe same Amazon Web Services account and Amazon Web Services Region. The\nname must have 1 to 32 characters. Valid characters are a-z, A-Z, 0-9, and\n: + = @ _ % - (hyphen). The name is not case sensitive.\n\nRegex Pattern: `^[a-zA-Z0-9](-*[a-zA-Z0-9]){0,31}$`", "type": "string" }, "tags": { "description": "An array of key-value pairs. You can use tags to categorize your Amazon Web\nServices resources in different ways, for example, by purpose, owner, or\nenvironment. For more information, see Tagging Amazon Web Services Resources\n(https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html).\n\nTags that you specify for the tuning job are also added to all training jobs\nthat the tuning job launches.", "items": { "description": "A tag object that consists of a key and an optional value, used to manage\nmetadata for SageMaker Amazon Web Services resources.\n\nYou can add tags to notebook instances, training jobs, hyperparameter tuning\njobs, batch transform jobs, models, labeling jobs, work teams, endpoint configurations,\nand endpoints. For more information on adding tags to SageMaker resources,\nsee AddTags (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_AddTags.html).\n\nFor more information on adding metadata to your Amazon Web Services resources\nwith tagging, see Tagging Amazon Web Services resources (https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html).\nFor advice on best practices for managing Amazon Web Services resources with\ntagging, see Tagging Best Practices: Implement an Effective Amazon Web Services\nResource Tagging Strategy (https://d1.awsstatic.com/whitepapers/aws-tagging-best-practices.pdf).", "properties": { "key": { "type": "string" }, "value": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "trainingJobDefinition": { "description": "The HyperParameterTrainingJobDefinition (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTrainingJobDefinition.html)\nobject that describes the training jobs that this tuning job launches, including\nstatic hyperparameters, input data configuration, output data configuration,\nresource configuration, and stopping condition.", "properties": { "algorithmSpecification": { "description": "Specifies which training algorithm to use for training jobs that a hyperparameter\ntuning job launches and the metrics to monitor.", "properties": { "algorithmName": { "type": "string" }, "metricDefinitions": { "items": { "description": "Specifies a metric that the training algorithm writes to stderr or stdout.\nYou can view these logs to understand how your training job performs and\ncheck for any errors encountered during training. SageMaker hyperparameter\ntuning captures all defined metrics. Specify one of the defined metrics to\nuse as an objective metric using the TuningObjective (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTrainingJobDefinition.html#sagemaker-Type-HyperParameterTrainingJobDefinition-TuningObjective)\nparameter in the HyperParameterTrainingJobDefinition API to evaluate job\nperformance during hyperparameter tuning.", "properties": { "name": { "type": "string" }, "regex": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "trainingImage": { "type": "string" }, "trainingInputMode": { "description": "The training input mode that the algorithm supports. For more information\nabout input modes, see Algorithms (https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html).\n\nPipe mode\n\nIf an algorithm supports Pipe mode, Amazon SageMaker streams data directly\nfrom Amazon S3 to the container.\n\nFile mode\n\nIf an algorithm supports File mode, SageMaker downloads the training data\nfrom S3 to the provisioned ML storage volume, and mounts the directory to\nthe Docker volume for the training container.\n\nYou must provision the ML storage volume with sufficient capacity to accommodate\nthe data downloaded from S3. In addition to the training data, the ML storage\nvolume also stores the output model. The algorithm container uses the ML\nstorage volume to also store intermediate information, if any.\n\nFor distributed algorithms, training data is distributed uniformly. Your\ntraining duration is predictable if the input data objects sizes are approximately\nthe same. SageMaker does not split the files any further for model training.\nIf the object sizes are skewed, training won't be optimal as the data distribution\nis also skewed when one host in a training cluster is overloaded, thus becoming\na bottleneck in training.\n\nFastFile mode\n\nIf an algorithm supports FastFile mode, SageMaker streams data directly from\nS3 to the container with no code changes, and provides file system access\nto the data. Users can author their training script to interact with these\nfiles as if they were stored on disk.\n\nFastFile mode works best when the data is read sequentially. Augmented manifest\nfiles aren't supported. The startup time is lower when there are fewer files\nin the S3 bucket provided.", "type": "string" } }, "type": "object", "additionalProperties": false }, "checkpointConfig": { "description": "Contains information about the output location for managed spot training\ncheckpoint data.", "properties": { "localPath": { "type": "string" }, "s3URI": { "type": "string" } }, "type": "object", "additionalProperties": false }, "definitionName": { "type": "string" }, "enableInterContainerTrafficEncryption": { "type": "boolean" }, "enableManagedSpotTraining": { "type": "boolean" }, "enableNetworkIsolation": { "type": "boolean" }, "hyperParameterRanges": { "description": "Specifies ranges of integer, continuous, and categorical hyperparameters\nthat a hyperparameter tuning job searches. The hyperparameter tuning job\nlaunches training jobs with hyperparameter values within these ranges to\nfind the combination of values that result in the training job with the best\nperformance as measured by the objective metric of the hyperparameter tuning\njob.\n\nThe maximum number of items specified for Array Members refers to the maximum\nnumber of hyperparameters for each range and also the maximum for the hyperparameter\ntuning job itself. That is, the sum of the number of hyperparameters for\nall the ranges can't exceed the maximum number specified.", "properties": { "autoParameters": { "items": { "description": "The name and an example value of the hyperparameter that you want to use\nin Autotune. If Automatic model tuning (AMT) determines that your hyperparameter\nis eligible for Autotune, an optimal hyperparameter range is selected for\nyou.", "properties": { "name": { "type": "string" }, "valueHint": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "categoricalParameterRanges": { "items": { "description": "A list of categorical hyperparameters to tune.", "properties": { "name": { "type": "string" }, "values": { "items": { "type": "string" }, "type": "array" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "continuousParameterRanges": { "items": { "description": "A list of continuous hyperparameters to tune.", "properties": { "maxValue": { "type": "string" }, "minValue": { "type": "string" }, "name": { "type": "string" }, "scalingType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "integerParameterRanges": { "items": { "description": "For a hyperparameter of the integer type, specifies the range that a hyperparameter\ntuning job searches.", "properties": { "maxValue": { "type": "string" }, "minValue": { "type": "string" }, "name": { "type": "string" }, "scalingType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" } }, "type": "object", "additionalProperties": false }, "inputDataConfig": { "items": { "description": "A channel is a named input source that training algorithms can consume.", "properties": { "channelName": { "type": "string" }, "compressionType": { "type": "string" }, "contentType": { "type": "string" }, "dataSource": { "description": "Describes the location of the channel data.", "properties": { "fileSystemDataSource": { "description": "Specifies a file system data source for a channel.", "properties": { "directoryPath": { "type": "string" }, "fileSystemAccessMode": { "type": "string" }, "fileSystemID": { "type": "string" }, "fileSystemType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "s3DataSource": { "description": "Describes the S3 data source.\n\nYour input bucket must be in the same Amazon Web Services region as your\ntraining job.", "properties": { "attributeNames": { "items": { "type": "string" }, "type": "array" }, "hubAccessConfig": { "description": "The configuration for a private hub model reference that points to a public\nSageMaker JumpStart model.\n\nFor more information about private hubs, see Private curated hubs for foundation\nmodel access control in JumpStart (https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-curated-hubs.html).", "properties": { "hubContentARN": { "type": "string" } }, "type": "object", "additionalProperties": false }, "instanceGroupNames": { "items": { "type": "string" }, "type": "array" }, "modelAccessConfig": { "description": "The access configuration file to control access to the ML model. You can\nexplicitly accept the model end-user license agreement (EULA) within the\nModelAccessConfig.\n\n * If you are a Jumpstart user, see the End-user license agreements (https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-choose.html#jumpstart-foundation-models-choose-eula)\n section for more details on accepting the EULA.\n\n * If you are an AutoML user, see the Optional Parameters section of Create\n an AutoML job to fine-tune text generation models using the API for details\n on How to set the EULA acceptance when fine-tuning a model using the AutoML\n API (https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-create-experiment-finetune-llms.html#autopilot-llms-finetuning-api-optional-params).", "properties": { "acceptEula": { "type": "boolean" } }, "type": "object", "additionalProperties": false }, "s3DataDistributionType": { "type": "string" }, "s3DataType": { "type": "string" }, "s3URI": { "type": "string" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "inputMode": { "description": "The training input mode that the algorithm supports. For more information\nabout input modes, see Algorithms (https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html).\n\nPipe mode\n\nIf an algorithm supports Pipe mode, Amazon SageMaker streams data directly\nfrom Amazon S3 to the container.\n\nFile mode\n\nIf an algorithm supports File mode, SageMaker downloads the training data\nfrom S3 to the provisioned ML storage volume, and mounts the directory to\nthe Docker volume for the training container.\n\nYou must provision the ML storage volume with sufficient capacity to accommodate\nthe data downloaded from S3. In addition to the training data, the ML storage\nvolume also stores the output model. The algorithm container uses the ML\nstorage volume to also store intermediate information, if any.\n\nFor distributed algorithms, training data is distributed uniformly. Your\ntraining duration is predictable if the input data objects sizes are approximately\nthe same. SageMaker does not split the files any further for model training.\nIf the object sizes are skewed, training won't be optimal as the data distribution\nis also skewed when one host in a training cluster is overloaded, thus becoming\na bottleneck in training.\n\nFastFile mode\n\nIf an algorithm supports FastFile mode, SageMaker streams data directly from\nS3 to the container with no code changes, and provides file system access\nto the data. Users can author their training script to interact with these\nfiles as if they were stored on disk.\n\nFastFile mode works best when the data is read sequentially. Augmented manifest\nfiles aren't supported. The startup time is lower when there are fewer files\nin the S3 bucket provided.", "type": "string" }, "recordWrapperType": { "type": "string" }, "shuffleConfig": { "description": "A configuration for a shuffle option for input data in a channel. If you\nuse S3Prefix for S3DataType, the results of the S3 key prefix matches are\nshuffled. If you use ManifestFile, the order of the S3 object references\nin the ManifestFile is shuffled. If you use AugmentedManifestFile, the order\nof the JSON lines in the AugmentedManifestFile is shuffled. The shuffling\norder is determined using the Seed value.\n\nFor Pipe input mode, when ShuffleConfig is specified shuffling is done at\nthe start of every epoch. With large datasets, this ensures that the order\nof the training data is different for each epoch, and it helps reduce bias\nand possible overfitting. In a multi-node training job when ShuffleConfig\nis combined with S3DataDistributionType of ShardedByS3Key, the data is shuffled\nacross nodes so that the content sent to a particular node on the first epoch\nmight be sent to a different node on the second epoch.", "properties": { "seed": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "type": "array" }, "outputDataConfig": { "description": "Provides information about how to store model training results (model artifacts).", "properties": { "compressionType": { "type": "string" }, "kmsKeyID": { "type": "string" }, "s3OutputPath": { "type": "string" } }, "type": "object", "additionalProperties": false }, "resourceConfig": { "description": "Describes the resources, including machine learning (ML) compute instances\nand ML storage volumes, to use for model training.", "properties": { "instanceCount": { "format": "int64", "type": "integer" }, "instanceGroups": { "items": { "description": "Defines an instance group for heterogeneous cluster training. When requesting\na training job using the CreateTrainingJob (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html)\nAPI, you can configure multiple instance groups .", "properties": { "instanceCount": { "format": "int64", "type": "integer" }, "instanceGroupName": { "type": "string" }, "instanceType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "instancePlacementConfig": { "description": "Configuration for how instances are placed and allocated within UltraServers.\nThis is only applicable for UltraServer capacity.", "properties": { "enableMultipleJobs": { "type": "boolean" }, "placementSpecifications": { "items": { "description": "Specifies how instances should be placed on a specific UltraServer.", "properties": { "instanceCount": { "format": "int64", "type": "integer" }, "ultraServerID": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" } }, "type": "object", "additionalProperties": false }, "instanceType": { "type": "string" }, "keepAlivePeriodInSeconds": { "description": "Optional. Customer requested period in seconds for which the Training cluster\nis kept alive after the job is finished.", "format": "int64", "type": "integer" }, "volumeKMSKeyID": { "type": "string" }, "volumeSizeInGB": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "retryStrategy": { "description": "The retry strategy to use when a training job fails due to an InternalServerError.\nRetryStrategy is specified as part of the CreateTrainingJob and CreateHyperParameterTuningJob\nrequests. You can add the StoppingCondition parameter to the request to limit\nthe training time for the complete job.", "properties": { "maximumRetryAttempts": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "roleARN": { "type": "string" }, "staticHyperParameters": { "additionalProperties": { "type": "string" }, "type": "object" }, "stoppingCondition": { "description": "Specifies a limit to how long a job can run. When the job reaches the time\nlimit, SageMaker ends the job. Use this API to cap costs.\n\nTo stop a training job, SageMaker sends the algorithm the SIGTERM signal,\nwhich delays job termination for 120 seconds. Algorithms can use this 120-second\nwindow to save the model artifacts, so the results of training are not lost.\n\nThe training algorithms provided by SageMaker automatically save the intermediate\nresults of a model training job when possible. This attempt to save artifacts\nis only a best effort case as model might not be in a state from which it\ncan be saved. For example, if training has just started, the model might\nnot be ready to save. When saved, this intermediate data is a valid model\nartifact. You can use it to create a model with CreateModel.\n\nThe Neural Topic Model (NTM) currently does not support saving intermediate\nmodel artifacts. When training NTMs, make sure that the maximum runtime is\nsufficient for the training job to complete.", "properties": { "maxPendingTimeInSeconds": { "description": "Maximum job scheduler pending time in seconds.", "format": "int64", "type": "integer" }, "maxRuntimeInSeconds": { "format": "int64", "type": "integer" }, "maxWaitTimeInSeconds": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "tuningObjective": { "description": "Defines the objective metric for a hyperparameter tuning job. Hyperparameter\ntuning uses the value of this metric to evaluate the training jobs it launches,\nand returns the training job that results in either the highest or lowest\nvalue for this metric, depending on the value you specify for the Type parameter.\nIf you want to define a custom objective metric, see Define metrics and environment\nvariables (https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-metrics-variables.html).", "properties": { "metricName": { "type": "string" }, "type_": { "type": "string" } }, "type": "object", "additionalProperties": false }, "vpcConfig": { "description": "Specifies an Amazon Virtual Private Cloud (VPC) that your SageMaker jobs,\nhosted models, and compute resources have access to. You can control access\nto and from your resources by configuring a VPC. For more information, see\nGive SageMaker Access to Resources in your Amazon VPC (https://docs.aws.amazon.com/sagemaker/latest/dg/infrastructure-give-access.html).", "properties": { "securityGroupIDs": { "items": { "type": "string" }, "type": "array" }, "subnets": { "items": { "type": "string" }, "type": "array" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "trainingJobDefinitions": { "description": "A list of the HyperParameterTrainingJobDefinition (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTrainingJobDefinition.html)\nobjects launched for this tuning job.", "items": { "description": "Defines the training jobs launched by a hyperparameter tuning job.", "properties": { "algorithmSpecification": { "description": "Specifies which training algorithm to use for training jobs that a hyperparameter\ntuning job launches and the metrics to monitor.", "properties": { "algorithmName": { "type": "string" }, "metricDefinitions": { "items": { "description": "Specifies a metric that the training algorithm writes to stderr or stdout.\nYou can view these logs to understand how your training job performs and\ncheck for any errors encountered during training. SageMaker hyperparameter\ntuning captures all defined metrics. Specify one of the defined metrics to\nuse as an objective metric using the TuningObjective (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTrainingJobDefinition.html#sagemaker-Type-HyperParameterTrainingJobDefinition-TuningObjective)\nparameter in the HyperParameterTrainingJobDefinition API to evaluate job\nperformance during hyperparameter tuning.", "properties": { "name": { "type": "string" }, "regex": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "trainingImage": { "type": "string" }, "trainingInputMode": { "description": "The training input mode that the algorithm supports. For more information\nabout input modes, see Algorithms (https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html).\n\nPipe mode\n\nIf an algorithm supports Pipe mode, Amazon SageMaker streams data directly\nfrom Amazon S3 to the container.\n\nFile mode\n\nIf an algorithm supports File mode, SageMaker downloads the training data\nfrom S3 to the provisioned ML storage volume, and mounts the directory to\nthe Docker volume for the training container.\n\nYou must provision the ML storage volume with sufficient capacity to accommodate\nthe data downloaded from S3. In addition to the training data, the ML storage\nvolume also stores the output model. The algorithm container uses the ML\nstorage volume to also store intermediate information, if any.\n\nFor distributed algorithms, training data is distributed uniformly. Your\ntraining duration is predictable if the input data objects sizes are approximately\nthe same. SageMaker does not split the files any further for model training.\nIf the object sizes are skewed, training won't be optimal as the data distribution\nis also skewed when one host in a training cluster is overloaded, thus becoming\na bottleneck in training.\n\nFastFile mode\n\nIf an algorithm supports FastFile mode, SageMaker streams data directly from\nS3 to the container with no code changes, and provides file system access\nto the data. Users can author their training script to interact with these\nfiles as if they were stored on disk.\n\nFastFile mode works best when the data is read sequentially. Augmented manifest\nfiles aren't supported. The startup time is lower when there are fewer files\nin the S3 bucket provided.", "type": "string" } }, "type": "object", "additionalProperties": false }, "checkpointConfig": { "description": "Contains information about the output location for managed spot training\ncheckpoint data.", "properties": { "localPath": { "type": "string" }, "s3URI": { "type": "string" } }, "type": "object", "additionalProperties": false }, "definitionName": { "type": "string" }, "enableInterContainerTrafficEncryption": { "type": "boolean" }, "enableManagedSpotTraining": { "type": "boolean" }, "enableNetworkIsolation": { "type": "boolean" }, "hyperParameterRanges": { "description": "Specifies ranges of integer, continuous, and categorical hyperparameters\nthat a hyperparameter tuning job searches. The hyperparameter tuning job\nlaunches training jobs with hyperparameter values within these ranges to\nfind the combination of values that result in the training job with the best\nperformance as measured by the objective metric of the hyperparameter tuning\njob.\n\nThe maximum number of items specified for Array Members refers to the maximum\nnumber of hyperparameters for each range and also the maximum for the hyperparameter\ntuning job itself. That is, the sum of the number of hyperparameters for\nall the ranges can't exceed the maximum number specified.", "properties": { "autoParameters": { "items": { "description": "The name and an example value of the hyperparameter that you want to use\nin Autotune. If Automatic model tuning (AMT) determines that your hyperparameter\nis eligible for Autotune, an optimal hyperparameter range is selected for\nyou.", "properties": { "name": { "type": "string" }, "valueHint": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "categoricalParameterRanges": { "items": { "description": "A list of categorical hyperparameters to tune.", "properties": { "name": { "type": "string" }, "values": { "items": { "type": "string" }, "type": "array" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "continuousParameterRanges": { "items": { "description": "A list of continuous hyperparameters to tune.", "properties": { "maxValue": { "type": "string" }, "minValue": { "type": "string" }, "name": { "type": "string" }, "scalingType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "integerParameterRanges": { "items": { "description": "For a hyperparameter of the integer type, specifies the range that a hyperparameter\ntuning job searches.", "properties": { "maxValue": { "type": "string" }, "minValue": { "type": "string" }, "name": { "type": "string" }, "scalingType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" } }, "type": "object", "additionalProperties": false }, "inputDataConfig": { "items": { "description": "A channel is a named input source that training algorithms can consume.", "properties": { "channelName": { "type": "string" }, "compressionType": { "type": "string" }, "contentType": { "type": "string" }, "dataSource": { "description": "Describes the location of the channel data.", "properties": { "fileSystemDataSource": { "description": "Specifies a file system data source for a channel.", "properties": { "directoryPath": { "type": "string" }, "fileSystemAccessMode": { "type": "string" }, "fileSystemID": { "type": "string" }, "fileSystemType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "s3DataSource": { "description": "Describes the S3 data source.\n\nYour input bucket must be in the same Amazon Web Services region as your\ntraining job.", "properties": { "attributeNames": { "items": { "type": "string" }, "type": "array" }, "hubAccessConfig": { "description": "The configuration for a private hub model reference that points to a public\nSageMaker JumpStart model.\n\nFor more information about private hubs, see Private curated hubs for foundation\nmodel access control in JumpStart (https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-curated-hubs.html).", "properties": { "hubContentARN": { "type": "string" } }, "type": "object", "additionalProperties": false }, "instanceGroupNames": { "items": { "type": "string" }, "type": "array" }, "modelAccessConfig": { "description": "The access configuration file to control access to the ML model. You can\nexplicitly accept the model end-user license agreement (EULA) within the\nModelAccessConfig.\n\n * If you are a Jumpstart user, see the End-user license agreements (https://docs.aws.amazon.com/sagemaker/latest/dg/jumpstart-foundation-models-choose.html#jumpstart-foundation-models-choose-eula)\n section for more details on accepting the EULA.\n\n * If you are an AutoML user, see the Optional Parameters section of Create\n an AutoML job to fine-tune text generation models using the API for details\n on How to set the EULA acceptance when fine-tuning a model using the AutoML\n API (https://docs.aws.amazon.com/sagemaker/latest/dg/autopilot-create-experiment-finetune-llms.html#autopilot-llms-finetuning-api-optional-params).", "properties": { "acceptEula": { "type": "boolean" } }, "type": "object", "additionalProperties": false }, "s3DataDistributionType": { "type": "string" }, "s3DataType": { "type": "string" }, "s3URI": { "type": "string" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "inputMode": { "description": "The training input mode that the algorithm supports. For more information\nabout input modes, see Algorithms (https://docs.aws.amazon.com/sagemaker/latest/dg/algos.html).\n\nPipe mode\n\nIf an algorithm supports Pipe mode, Amazon SageMaker streams data directly\nfrom Amazon S3 to the container.\n\nFile mode\n\nIf an algorithm supports File mode, SageMaker downloads the training data\nfrom S3 to the provisioned ML storage volume, and mounts the directory to\nthe Docker volume for the training container.\n\nYou must provision the ML storage volume with sufficient capacity to accommodate\nthe data downloaded from S3. In addition to the training data, the ML storage\nvolume also stores the output model. The algorithm container uses the ML\nstorage volume to also store intermediate information, if any.\n\nFor distributed algorithms, training data is distributed uniformly. Your\ntraining duration is predictable if the input data objects sizes are approximately\nthe same. SageMaker does not split the files any further for model training.\nIf the object sizes are skewed, training won't be optimal as the data distribution\nis also skewed when one host in a training cluster is overloaded, thus becoming\na bottleneck in training.\n\nFastFile mode\n\nIf an algorithm supports FastFile mode, SageMaker streams data directly from\nS3 to the container with no code changes, and provides file system access\nto the data. Users can author their training script to interact with these\nfiles as if they were stored on disk.\n\nFastFile mode works best when the data is read sequentially. Augmented manifest\nfiles aren't supported. The startup time is lower when there are fewer files\nin the S3 bucket provided.", "type": "string" }, "recordWrapperType": { "type": "string" }, "shuffleConfig": { "description": "A configuration for a shuffle option for input data in a channel. If you\nuse S3Prefix for S3DataType, the results of the S3 key prefix matches are\nshuffled. If you use ManifestFile, the order of the S3 object references\nin the ManifestFile is shuffled. If you use AugmentedManifestFile, the order\nof the JSON lines in the AugmentedManifestFile is shuffled. The shuffling\norder is determined using the Seed value.\n\nFor Pipe input mode, when ShuffleConfig is specified shuffling is done at\nthe start of every epoch. With large datasets, this ensures that the order\nof the training data is different for each epoch, and it helps reduce bias\nand possible overfitting. In a multi-node training job when ShuffleConfig\nis combined with S3DataDistributionType of ShardedByS3Key, the data is shuffled\nacross nodes so that the content sent to a particular node on the first epoch\nmight be sent to a different node on the second epoch.", "properties": { "seed": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "type": "array" }, "outputDataConfig": { "description": "Provides information about how to store model training results (model artifacts).", "properties": { "compressionType": { "type": "string" }, "kmsKeyID": { "type": "string" }, "s3OutputPath": { "type": "string" } }, "type": "object", "additionalProperties": false }, "resourceConfig": { "description": "Describes the resources, including machine learning (ML) compute instances\nand ML storage volumes, to use for model training.", "properties": { "instanceCount": { "format": "int64", "type": "integer" }, "instanceGroups": { "items": { "description": "Defines an instance group for heterogeneous cluster training. When requesting\na training job using the CreateTrainingJob (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html)\nAPI, you can configure multiple instance groups .", "properties": { "instanceCount": { "format": "int64", "type": "integer" }, "instanceGroupName": { "type": "string" }, "instanceType": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "instancePlacementConfig": { "description": "Configuration for how instances are placed and allocated within UltraServers.\nThis is only applicable for UltraServer capacity.", "properties": { "enableMultipleJobs": { "type": "boolean" }, "placementSpecifications": { "items": { "description": "Specifies how instances should be placed on a specific UltraServer.", "properties": { "instanceCount": { "format": "int64", "type": "integer" }, "ultraServerID": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" } }, "type": "object", "additionalProperties": false }, "instanceType": { "type": "string" }, "keepAlivePeriodInSeconds": { "description": "Optional. Customer requested period in seconds for which the Training cluster\nis kept alive after the job is finished.", "format": "int64", "type": "integer" }, "volumeKMSKeyID": { "type": "string" }, "volumeSizeInGB": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "retryStrategy": { "description": "The retry strategy to use when a training job fails due to an InternalServerError.\nRetryStrategy is specified as part of the CreateTrainingJob and CreateHyperParameterTuningJob\nrequests. You can add the StoppingCondition parameter to the request to limit\nthe training time for the complete job.", "properties": { "maximumRetryAttempts": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "roleARN": { "type": "string" }, "staticHyperParameters": { "additionalProperties": { "type": "string" }, "type": "object" }, "stoppingCondition": { "description": "Specifies a limit to how long a job can run. When the job reaches the time\nlimit, SageMaker ends the job. Use this API to cap costs.\n\nTo stop a training job, SageMaker sends the algorithm the SIGTERM signal,\nwhich delays job termination for 120 seconds. Algorithms can use this 120-second\nwindow to save the model artifacts, so the results of training are not lost.\n\nThe training algorithms provided by SageMaker automatically save the intermediate\nresults of a model training job when possible. This attempt to save artifacts\nis only a best effort case as model might not be in a state from which it\ncan be saved. For example, if training has just started, the model might\nnot be ready to save. When saved, this intermediate data is a valid model\nartifact. You can use it to create a model with CreateModel.\n\nThe Neural Topic Model (NTM) currently does not support saving intermediate\nmodel artifacts. When training NTMs, make sure that the maximum runtime is\nsufficient for the training job to complete.", "properties": { "maxPendingTimeInSeconds": { "description": "Maximum job scheduler pending time in seconds.", "format": "int64", "type": "integer" }, "maxRuntimeInSeconds": { "format": "int64", "type": "integer" }, "maxWaitTimeInSeconds": { "format": "int64", "type": "integer" } }, "type": "object", "additionalProperties": false }, "tuningObjective": { "description": "Defines the objective metric for a hyperparameter tuning job. Hyperparameter\ntuning uses the value of this metric to evaluate the training jobs it launches,\nand returns the training job that results in either the highest or lowest\nvalue for this metric, depending on the value you specify for the Type parameter.\nIf you want to define a custom objective metric, see Define metrics and environment\nvariables (https://docs.aws.amazon.com/sagemaker/latest/dg/automatic-model-tuning-define-metrics-variables.html).", "properties": { "metricName": { "type": "string" }, "type_": { "type": "string" } }, "type": "object", "additionalProperties": false }, "vpcConfig": { "description": "Specifies an Amazon Virtual Private Cloud (VPC) that your SageMaker jobs,\nhosted models, and compute resources have access to. You can control access\nto and from your resources by configuring a VPC. For more information, see\nGive SageMaker Access to Resources in your Amazon VPC (https://docs.aws.amazon.com/sagemaker/latest/dg/infrastructure-give-access.html).", "properties": { "securityGroupIDs": { "items": { "type": "string" }, "type": "array" }, "subnets": { "items": { "type": "string" }, "type": "array" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false }, "type": "array" }, "warmStartConfig": { "description": "Specifies the configuration for starting the hyperparameter tuning job using\none or more previous tuning jobs as a starting point. The results of previous\ntuning jobs are used to inform which combinations of hyperparameters to search\nover in the new tuning job.\n\nAll training jobs launched by the new hyperparameter tuning job are evaluated\nby using the objective metric. If you specify IDENTICAL_DATA_AND_ALGORITHM\nas the WarmStartType value for the warm start configuration, the training\njob that performs the best in the new tuning job is compared to the best\ntraining jobs from the parent tuning jobs. From these, the training job that\nperforms the best as measured by the objective metric is returned as the\noverall best training job.\n\nAll training jobs launched by parent hyperparameter tuning jobs and the new\nhyperparameter tuning jobs count against the limit of training jobs for the\ntuning job.", "properties": { "parentHyperParameterTuningJobs": { "items": { "description": "A previously completed or stopped hyperparameter tuning job to be used as\na starting point for a new hyperparameter tuning job.", "properties": { "hyperParameterTuningJobName": { "type": "string" } }, "type": "object", "additionalProperties": false }, "type": "array" }, "warmStartType": { "type": "string" } }, "type": "object", "additionalProperties": false } }, "required": [ "hyperParameterTuningJobConfig", "hyperParameterTuningJobName" ], "type": "object", "additionalProperties": false }, "status": { "description": "HyperParameterTuningJobStatus defines the observed state of HyperParameterTuningJob", "properties": { "ackResourceMetadata": { "description": "All CRs managed by ACK have a common `Status.ACKResourceMetadata` member\nthat is used to contain resource sync state, account ownership,\nconstructed ARN for the resource", "properties": { "arn": { "description": "ARN is the Amazon Resource Name for the resource. This is a\nglobally-unique identifier and is set only by the ACK service controller\nonce the controller has orchestrated the creation of the resource OR\nwhen it has verified that an \"adopted\" resource (a resource where the\nARN annotation was set by the Kubernetes user on the CR) exists and\nmatches the supplied CR's Spec field values.\nhttps://github.com/aws/aws-controllers-k8s/issues/270", "type": "string" }, "ownerAccountID": { "description": "OwnerAccountID is the AWS Account ID of the account that owns the\nbackend AWS service API resource.", "type": "string" }, "region": { "description": "Region is the AWS region in which the resource exists or will exist.", "type": "string" } }, "required": [ "ownerAccountID", "region" ], "type": "object", "additionalProperties": false }, "bestTrainingJob": { "description": "A TrainingJobSummary (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TrainingJobSummary.html)\nobject that describes the training job that completed with the best current\nHyperParameterTuningJobObjective (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobObjective.html).", "properties": { "creationTime": { "format": "date-time", "type": "string" }, "failureReason": { "type": "string" }, "finalHyperParameterTuningJobObjectiveMetric": { "description": "Shows the latest objective metric emitted by a training job that was launched\nby a hyperparameter tuning job. You define the objective metric in the HyperParameterTuningJobObjective\nparameter of HyperParameterTuningJobConfig (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobConfig.html).", "properties": { "metricName": { "type": "string" }, "type_": { "type": "string" }, "value": { "type": "number" } }, "type": "object", "additionalProperties": false }, "objectiveStatus": { "type": "string" }, "trainingEndTime": { "format": "date-time", "type": "string" }, "trainingJobARN": { "type": "string" }, "trainingJobDefinitionName": { "type": "string" }, "trainingJobName": { "type": "string" }, "trainingJobStatus": { "type": "string" }, "trainingStartTime": { "format": "date-time", "type": "string" }, "tunedHyperParameters": { "additionalProperties": { "type": "string" }, "type": "object" }, "tuningJobName": { "type": "string" } }, "type": "object", "additionalProperties": false }, "conditions": { "description": "All CRs managed by ACK have a common `Status.Conditions` member that\ncontains a collection of `ackv1alpha1.Condition` objects that describe\nthe various terminal states of the CR and its backend AWS service API\nresource", "items": { "description": "Condition is the common struct used by all CRDs managed by ACK service\ncontrollers to indicate terminal states of the CR and its backend AWS\nservice API resource", "properties": { "lastTransitionTime": { "description": "Last time the condition transitioned from one status to another.", "format": "date-time", "type": "string" }, "message": { "description": "A human readable message indicating details about the transition.", "type": "string" }, "reason": { "description": "The reason for the condition's last transition.", "type": "string" }, "status": { "description": "Status of the condition, one of True, False, Unknown.", "type": "string" }, "type": { "description": "Type is the type of the Condition", "type": "string" } }, "required": [ "status", "type" ], "type": "object", "additionalProperties": false }, "type": "array" }, "failureReason": { "description": "If the tuning job failed, the reason it failed.", "type": "string" }, "hyperParameterTuningJobStatus": { "description": "The status of the tuning job.", "type": "string" }, "overallBestTrainingJob": { "description": "If the hyperparameter tuning job is an warm start tuning job with a WarmStartType\nof IDENTICAL_DATA_AND_ALGORITHM, this is the TrainingJobSummary (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_TrainingJobSummary.html)\nfor the training job with the best objective metric value of all training\njobs launched by this tuning job and all parent jobs specified for the warm\nstart tuning job.", "properties": { "creationTime": { "format": "date-time", "type": "string" }, "failureReason": { "type": "string" }, "finalHyperParameterTuningJobObjectiveMetric": { "description": "Shows the latest objective metric emitted by a training job that was launched\nby a hyperparameter tuning job. You define the objective metric in the HyperParameterTuningJobObjective\nparameter of HyperParameterTuningJobConfig (https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_HyperParameterTuningJobConfig.html).", "properties": { "metricName": { "type": "string" }, "type_": { "type": "string" }, "value": { "type": "number" } }, "type": "object", "additionalProperties": false }, "objectiveStatus": { "type": "string" }, "trainingEndTime": { "format": "date-time", "type": "string" }, "trainingJobARN": { "type": "string" }, "trainingJobDefinitionName": { "type": "string" }, "trainingJobName": { "type": "string" }, "trainingJobStatus": { "type": "string" }, "trainingStartTime": { "format": "date-time", "type": "string" }, "tunedHyperParameters": { "additionalProperties": { "type": "string" }, "type": "object" }, "tuningJobName": { "type": "string" } }, "type": "object", "additionalProperties": false } }, "type": "object", "additionalProperties": false } }, "type": "object" }