awscdkkinesisanalyticsflinkalpha

package module
v2.184.1-alpha.0 Latest Latest
Warning

This package is not in the latest version of its module.

 Go to latest Published: Mar 14, 2025 License: Apache-2.0 Imports: 14 Imported by: 0

README

---

The APIs of higher level constructs in this module are experimental and under active development. They are subject to non-backward compatible changes or removal in any future version. These are not subject to the Semantic Versioning model and breaking changes will be announced in the release notes. This means that while you may use them, you may need to update your source code when upgrading to a newer version of this package.

This package provides constructs for creating Kinesis Analytics Flink applications. To learn more about using using managed Flink applications, see the AWS developer guide.

To create a new Flink application, use the Application construct:

import path "github.com/aws-samples/dummy/path"
import cloudwatch "github.com/aws/aws-cdk-go/awscdk"
import "github.com/aws/aws-cdk-go/awscdk"
import integ "github.com/aws/aws-cdk-go/awscdkintegtestsalpha"
import "github.com/aws/aws-cdk-go/awscdkkinesisanalyticsflinkalpha"

app := core.NewApp()
stack := core.NewStack(app, jsii.String("FlinkAppTest"))

flinkRuntimes := []runtime{
	flink.runtime_FLINK_1_6(),
	flink.runtime_FLINK_1_8(),
	flink.runtime_FLINK_1_11(),
	flink.runtime_FLINK_1_13(),
	flink.runtime_FLINK_1_15(),
	flink.runtime_FLINK_1_18(),
	flink.runtime_FLINK_1_19(),
	flink.runtime_FLINK_1_20(),
}

flinkRuntimes.forEach((runtime) => {
  const flinkApp = new flink.Application(stack, `App-${runtime.value}`, {
    code: flink.ApplicationCode.fromAsset(path.join(__dirname, 'code-asset')),
    runtime: runtime,
  });

  new cloudwatch.Alarm(stack, `Alarm-${runtime.value}`, {
    metric: flinkApp.metricFullRestarts(),
    evaluationPeriods: 1,
    threshold: 3,
  });
})

integ.NewIntegTest(app, jsii.String("ApplicationTest"), &IntegTestProps{
	TestCases: []stack{
		stack,
	},
})

The code property can use fromAsset as shown above to reference a local jar file in s3 or fromBucket to reference a file in s3.

flink.NewApplication(stack, jsii.String("App"), &ApplicationProps{
	Code: flink.ApplicationCode_FromBucket(bucket, fileKey),
	Runtime: flink.Runtime_FLINK_1_19(),
})

The propertyGroups property provides a way of passing arbitrary runtime properties to your Flink application. You can use the aws-kinesisanalytics-runtime library to retrieve these properties.

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	PropertyGroups: map[string]map[string]*string{
		"FlinkApplicationProperties": map[string]*string{
			"inputStreamName": jsii.String("my-input-kinesis-stream"),
			"outputStreamName": jsii.String("my-output-kinesis-stream"),
		},
	},
	// ...
	Runtime: flink.Runtime_FLINK_1_20(),
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
})

Flink applications also have specific configuration for passing parameters when the Flink job starts. These include parameters for checkpointing, snapshotting, monitoring, and parallelism.

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
	Runtime: flink.Runtime_FLINK_1_20(),
	CheckpointingEnabled: jsii.Boolean(true),
	 // default is true
	CheckpointInterval: awscdk.Duration_Seconds(jsii.Number(30)),
	 // default is 1 minute
	MinPauseBetweenCheckpoints: awscdk.Duration_*Seconds(jsii.Number(10)),
	 // default is 5 seconds
	LogLevel: flink.LogLevel_ERROR,
	 // default is INFO
	MetricsLevel: flink.MetricsLevel_PARALLELISM,
	 // default is APPLICATION
	AutoScalingEnabled: jsii.Boolean(false),
	 // default is true
	Parallelism: jsii.Number(32),
	 // default is 1
	ParallelismPerKpu: jsii.Number(2),
	 // default is 1
	SnapshotsEnabled: jsii.Boolean(false),
	 // default is true
	LogGroup: logs.NewLogGroup(this, jsii.String("LogGroup")),
})

Flink applications can optionally be deployed in a VPC:

var bucket bucket
var vpc vpc

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
	Runtime: flink.Runtime_FLINK_1_20(),
	Vpc: Vpc,
})

Documentation

Overview

A CDK Construct Library for Kinesis Analytics Flink applications

Index

Constants

This section is empty.

Variables

This section is empty.

Functions

func Application_IsConstruct 

func Application_IsConstruct(x interface{}) *bool

Checks if `x` is a construct.

Use this method instead of `instanceof` to properly detect `Construct` instances, even when the construct library is symlinked.

Explanation: in JavaScript, multiple copies of the `constructs` library on disk are seen as independent, completely different libraries. As a consequence, the class `Construct` in each copy of the `constructs` library is seen as a different class, and an instance of one class will not test as `instanceof` the other class. `npm install` will not create installations like this, but users may manually symlink construct libraries together or use a monorepo tool: in those cases, multiple copies of the `constructs` library can be accidentally installed, and `instanceof` will behave unpredictably. It is safest to avoid using `instanceof`, and using this type-testing method instead.

Returns: true if `x` is an object created from a class which extends `Construct`. Experimental.

func Application_IsOwnedResource 

func Application_IsOwnedResource(construct constructs.IConstruct) *bool

Returns true if the construct was created by CDK, and false otherwise. Experimental.

func Application_IsResource 

func Application_IsResource(construct constructs.IConstruct) *bool

Check whether the given construct is a Resource. Experimental.

func NewApplicationCode_Override 

func NewApplicationCode_Override(a ApplicationCode)

Experimental.

func NewApplication_Override 

func NewApplication_Override(a Application, scope constructs.Construct, id *string, props *ApplicationProps)

Experimental.

Types

type Application 

type Application interface {
	awscdk.Resource
	IApplication
	// The application ARN.
	// Experimental.
	ApplicationArn() *string
	// The name of the Flink application.
	// Experimental.
	ApplicationName() *string
	// The network connections associated with this resource.
	// Experimental.
	Connections() awsec2.Connections
	// The environment this resource belongs to.
	//
	// For resources that are created and managed by the CDK
	// (generally, those created by creating new class instances like Role, Bucket, etc.),
	// this is always the same as the environment of the stack they belong to;
	// however, for imported resources
	// (those obtained from static methods like fromRoleArn, fromBucketName, etc.),
	// that might be different than the stack they were imported into.
	// Experimental.
	Env() *awscdk.ResourceEnvironment
	// The principal to grant permissions to.
	// Experimental.
	GrantPrincipal() awsiam.IPrincipal
	// The tree node.
	// Experimental.
	Node() constructs.Node
	// Returns a string-encoded token that resolves to the physical name that should be passed to the CloudFormation resource.
	//
	// This value will resolve to one of the following:
	// - a concrete value (e.g. `"my-awesome-bucket"`)
	// - `undefined`, when a name should be generated by CloudFormation
	// - a concrete name generated automatically during synthesis, in
	//   cross-environment scenarios.
	// Experimental.
	PhysicalName() *string
	// The application IAM role.
	// Experimental.
	Role() awsiam.IRole
	// The stack in which this resource is defined.
	// Experimental.
	Stack() awscdk.Stack
	// Implement the convenience `IApplication.addToPrincipalPolicy` method.
	// Experimental.
	AddToRolePolicy(policyStatement awsiam.PolicyStatement) *bool
	// Apply the given removal policy to this resource.
	//
	// The Removal Policy controls what happens to this resource when it stops
	// being managed by CloudFormation, either because you've removed it from the
	// CDK application or because you've made a change that requires the resource
	// to be replaced.
	//
	// The resource can be deleted (`RemovalPolicy.DESTROY`), or left in your AWS
	// account for data recovery and cleanup later (`RemovalPolicy.RETAIN`).
	// Experimental.
	ApplyRemovalPolicy(policy awscdk.RemovalPolicy)
	// Experimental.
	GeneratePhysicalName() *string
	// Returns an environment-sensitive token that should be used for the resource's "ARN" attribute (e.g. `bucket.bucketArn`).
	//
	// Normally, this token will resolve to `arnAttr`, but if the resource is
	// referenced across environments, `arnComponents` will be used to synthesize
	// a concrete ARN with the resource's physical name. Make sure to reference
	// `this.physicalName` in `arnComponents`.
	// Experimental.
	GetResourceArnAttribute(arnAttr *string, arnComponents *awscdk.ArnComponents) *string
	// Returns an environment-sensitive token that should be used for the resource's "name" attribute (e.g. `bucket.bucketName`).
	//
	// Normally, this token will resolve to `nameAttr`, but if the resource is
	// referenced across environments, it will be resolved to `this.physicalName`,
	// which will be a concrete name.
	// Experimental.
	GetResourceNameAttribute(nameAttr *string) *string
	// Return a CloudWatch metric associated with this Flink application.
	// Experimental.
	Metric(metricName *string, props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time (in milliseconds) this task or operator is back pressured per second.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricBackPressuredTimeMsPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time (in milliseconds) this task or operator is busy (neither idle nor back pressured) per second.
	//
	// Can be NaN, if the value could not be
	// calculated.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricBusyTimePerMsPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The overall percentage of CPU utilization across task managers.
	//
	// For
	// example, if there are five task managers, Kinesis Data Analytics publishes
	// five samples of this metric per reporting interval.
	//
	// Units: Percentage
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricCpuUtilization(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The last watermark this application/operator/task/thread has received.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricCurrentInputWatermark(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The last watermark this application/operator/task/thread has received.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricCurrentOutputWatermark(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time elapsed during an outage for failing/recovering jobs.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricDowntime(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of times this job has fully restarted since it was submitted.
	//
	// This metric does not measure fine-grained restarts.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricFullRestarts(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Overall heap memory utilization across task managers.
	//
	// For example, if there
	// are five task managers, Kinesis Data Analytics publishes five samples of
	// this metric per reporting interval.
	//
	// Units: Percentage
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricHeapMemoryUtilization(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time (in milliseconds) this task or operator is idle (has no data to process) per second.
	//
	// Idle time excludes back pressured time, so if the task
	// is back pressured it is not idle.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricIdleTimeMsPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The number of Kinesis Processing Units that are used to run your stream processing application.
	//
	// The average number of KPUs used each hour
	// determines the billing for your application.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricKpus(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time it took to complete the last checkpoint.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricLastCheckpointDuration(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total size of the last checkpoint.
	//
	// Units: Bytes
	//
	// Reporting Level: Application.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricLastCheckpointSize(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total amount of managed memory.
	//
	// Units: Bytes
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricManagedMemoryTotal(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The amount of managed memory currently used.
	//
	// Units: Bytes
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricManagedMemoryUsed(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Derived from managedMemoryUsed/managedMemoryTotal.
	//
	// Units: Percentage
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricManagedMemoryUtilization(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The number of times checkpointing has failed.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricNumberOfFailedCheckpoints(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The number of records this operator or task has dropped due to arriving late.
	//
	// Units: Count
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricNumLateRecordsDropped(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator, or task has received.
	//
	// Units: Count
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsIn(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator or task has received per second.
	//
	// Units: Count/Second
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsInPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator or task has emitted.
	//
	// Units: Count
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsOut(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator or task has emitted per second.
	//
	// Units: Count/Second
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsOutPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of old garbage collection operations that have occurred across all task managers.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricOldGenerationGCCount(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total time spent performing old garbage collection operations.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricOldGenerationGCTime(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of live threads used by the application.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricThreadsCount(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time that the job has been running without interruption.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricUptime(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Returns a string representation of this construct.
	// Experimental.
	ToString() *string
}

The L2 construct for Flink Kinesis Data Applications.

Example: 

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	PropertyGroups: map[string]map[string]*string{
		"FlinkApplicationProperties": map[string]*string{
			"inputStreamName": jsii.String("my-input-kinesis-stream"),
			"outputStreamName": jsii.String("my-output-kinesis-stream"),
		},
	},
	// ...
	Runtime: flink.Runtime_FLINK_1_20(),
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
})

Experimental.

func NewApplication 

func NewApplication(scope constructs.Construct, id *string, props *ApplicationProps) Application

Experimental.

type ApplicationAttributes 

type ApplicationAttributes struct {
	// The ARN of the Flink application.
	//
	// Format: arn:<partition>:kinesisanalytics:<region>:<account-id>:application/<application-name>.
	// Experimental.
	ApplicationArn *string `field:"required" json:"applicationArn" yaml:"applicationArn"`
	// The security groups for this Flink application if deployed in a VPC.
	// Default: - no security groups.
	//
	// Experimental.
	SecurityGroups *[]awsec2.ISecurityGroup `field:"optional" json:"securityGroups" yaml:"securityGroups"`
}

Attributes used for importing an Application with Application.fromApplicationAttributes.

Example: 

// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
import kinesisanalytics_flink_alpha "github.com/aws/aws-cdk-go/awscdkkinesisanalyticsflinkalpha"
import "github.com/aws/aws-cdk-go/awscdk"

var securityGroup securityGroup

applicationAttributes := &ApplicationAttributes{
	ApplicationArn: jsii.String("applicationArn"),

	// the properties below are optional
	SecurityGroups: []iSecurityGroup{
		securityGroup,
	},
}

Experimental.

type ApplicationCode 

type ApplicationCode interface {
	// A method to lazily bind asset resources to the parent FlinkApplication.
	// Experimental.
	Bind(scope constructs.Construct) *ApplicationCodeConfig
}

Code configuration providing the location to a Flink application JAR file.

Example: 

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	PropertyGroups: map[string]map[string]*string{
		"FlinkApplicationProperties": map[string]*string{
			"inputStreamName": jsii.String("my-input-kinesis-stream"),
			"outputStreamName": jsii.String("my-output-kinesis-stream"),
		},
	},
	// ...
	Runtime: flink.Runtime_FLINK_1_20(),
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
})

Experimental.

func ApplicationCode_FromAsset 

func ApplicationCode_FromAsset(path *string, options *awss3assets.AssetOptions) ApplicationCode

Reference code from a local directory containing a Flink JAR file. Experimental.

func ApplicationCode_FromBucket 

func ApplicationCode_FromBucket(bucket awss3.IBucket, fileKey *string, objectVersion *string) ApplicationCode

Reference code from an S3 bucket. Experimental.

type ApplicationCodeConfig 

type ApplicationCodeConfig struct {
	// Low-level Cloudformation ApplicationConfigurationProperty.
	// Experimental.
	ApplicationCodeConfigurationProperty *awskinesisanalytics.CfnApplicationV2_ApplicationConfigurationProperty `field:"required" json:"applicationCodeConfigurationProperty" yaml:"applicationCodeConfigurationProperty"`
	// S3 Bucket that stores the Flink application code.
	// Experimental.
	Bucket awss3.IBucket `field:"required" json:"bucket" yaml:"bucket"`
}

The return type of `ApplicationCode.bind`. This represents CloudFormation configuration and an s3 bucket holding the Flink application JAR file.

Example: 

// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
import kinesisanalytics_flink_alpha "github.com/aws/aws-cdk-go/awscdkkinesisanalyticsflinkalpha"
import "github.com/aws/aws-cdk-go/awscdk"

var bucket bucket

applicationCodeConfig := &ApplicationCodeConfig{
	ApplicationCodeConfigurationProperty: &ApplicationConfigurationProperty{
		ApplicationCodeConfiguration: &ApplicationCodeConfigurationProperty{
			CodeContent: &CodeContentProperty{
				S3ContentLocation: &S3ContentLocationProperty{
					BucketArn: jsii.String("bucketArn"),
					FileKey: jsii.String("fileKey"),

					// the properties below are optional
					ObjectVersion: jsii.String("objectVersion"),
				},
				TextContent: jsii.String("textContent"),
				ZipFileContent: jsii.String("zipFileContent"),
			},
			CodeContentType: jsii.String("codeContentType"),
		},
		ApplicationSnapshotConfiguration: &ApplicationSnapshotConfigurationProperty{
			SnapshotsEnabled: jsii.Boolean(false),
		},
		ApplicationSystemRollbackConfiguration: &ApplicationSystemRollbackConfigurationProperty{
			RollbackEnabled: jsii.Boolean(false),
		},
		EnvironmentProperties: &EnvironmentPropertiesProperty{
			PropertyGroups: []interface{}{
				&PropertyGroupProperty{
					PropertyGroupId: jsii.String("propertyGroupId"),
					PropertyMap: map[string]*string{
						"propertyMapKey": jsii.String("propertyMap"),
					},
				},
			},
		},
		FlinkApplicationConfiguration: &FlinkApplicationConfigurationProperty{
			CheckpointConfiguration: &CheckpointConfigurationProperty{
				ConfigurationType: jsii.String("configurationType"),

				// the properties below are optional
				CheckpointingEnabled: jsii.Boolean(false),
				CheckpointInterval: jsii.Number(123),
				MinPauseBetweenCheckpoints: jsii.Number(123),
			},
			MonitoringConfiguration: &MonitoringConfigurationProperty{
				ConfigurationType: jsii.String("configurationType"),

				// the properties below are optional
				LogLevel: jsii.String("logLevel"),
				MetricsLevel: jsii.String("metricsLevel"),
			},
			ParallelismConfiguration: &ParallelismConfigurationProperty{
				ConfigurationType: jsii.String("configurationType"),

				// the properties below are optional
				AutoScalingEnabled: jsii.Boolean(false),
				Parallelism: jsii.Number(123),
				ParallelismPerKpu: jsii.Number(123),
			},
		},
		SqlApplicationConfiguration: &SqlApplicationConfigurationProperty{
			Inputs: []interface{}{
				&InputProperty{
					InputSchema: &InputSchemaProperty{
						RecordColumns: []interface{}{
							&RecordColumnProperty{
								Name: jsii.String("name"),
								SqlType: jsii.String("sqlType"),

								// the properties below are optional
								Mapping: jsii.String("mapping"),
							},
						},
						RecordFormat: &RecordFormatProperty{
							RecordFormatType: jsii.String("recordFormatType"),

							// the properties below are optional
							MappingParameters: &MappingParametersProperty{
								CsvMappingParameters: &CSVMappingParametersProperty{
									RecordColumnDelimiter: jsii.String("recordColumnDelimiter"),
									RecordRowDelimiter: jsii.String("recordRowDelimiter"),
								},
								JsonMappingParameters: &JSONMappingParametersProperty{
									RecordRowPath: jsii.String("recordRowPath"),
								},
							},
						},

						// the properties below are optional
						RecordEncoding: jsii.String("recordEncoding"),
					},
					NamePrefix: jsii.String("namePrefix"),

					// the properties below are optional
					InputParallelism: &InputParallelismProperty{
						Count: jsii.Number(123),
					},
					InputProcessingConfiguration: &InputProcessingConfigurationProperty{
						InputLambdaProcessor: &InputLambdaProcessorProperty{
							ResourceArn: jsii.String("resourceArn"),
						},
					},
					KinesisFirehoseInput: &KinesisFirehoseInputProperty{
						ResourceArn: jsii.String("resourceArn"),
					},
					KinesisStreamsInput: &KinesisStreamsInputProperty{
						ResourceArn: jsii.String("resourceArn"),
					},
				},
			},
		},
		VpcConfigurations: []interface{}{
			&VpcConfigurationProperty{
				SecurityGroupIds: []*string{
					jsii.String("securityGroupIds"),
				},
				SubnetIds: []*string{
					jsii.String("subnetIds"),
				},
			},
		},
		ZeppelinApplicationConfiguration: &ZeppelinApplicationConfigurationProperty{
			CatalogConfiguration: &CatalogConfigurationProperty{
				GlueDataCatalogConfiguration: &GlueDataCatalogConfigurationProperty{
					DatabaseArn: jsii.String("databaseArn"),
				},
			},
			CustomArtifactsConfiguration: []interface{}{
				&CustomArtifactConfigurationProperty{
					ArtifactType: jsii.String("artifactType"),

					// the properties below are optional
					MavenReference: &MavenReferenceProperty{
						ArtifactId: jsii.String("artifactId"),
						GroupId: jsii.String("groupId"),
						Version: jsii.String("version"),
					},
					S3ContentLocation: &S3ContentLocationProperty{
						BucketArn: jsii.String("bucketArn"),
						FileKey: jsii.String("fileKey"),

						// the properties below are optional
						ObjectVersion: jsii.String("objectVersion"),
					},
				},
			},
			DeployAsApplicationConfiguration: &DeployAsApplicationConfigurationProperty{
				S3ContentLocation: &S3ContentBaseLocationProperty{
					BucketArn: jsii.String("bucketArn"),

					// the properties below are optional
					BasePath: jsii.String("basePath"),
				},
			},
			MonitoringConfiguration: &ZeppelinMonitoringConfigurationProperty{
				LogLevel: jsii.String("logLevel"),
			},
		},
	},
	Bucket: bucket,
}

Experimental.

type ApplicationProps 

type ApplicationProps struct {
	// The Flink code asset to run.
	// Experimental.
	Code ApplicationCode `field:"required" json:"code" yaml:"code"`
	// The Flink version to use for this application.
	// Experimental.
	Runtime Runtime `field:"required" json:"runtime" yaml:"runtime"`
	// A name for your Application that is unique to an AWS account.
	// Default: - CloudFormation-generated name.
	//
	// Experimental.
	ApplicationName *string `field:"optional" json:"applicationName" yaml:"applicationName"`
	// Whether the Kinesis Data Analytics service can increase the parallelism of the application in response to resource usage.
	// Default: true.
	//
	// Experimental.
	AutoScalingEnabled *bool `field:"optional" json:"autoScalingEnabled" yaml:"autoScalingEnabled"`
	// Whether checkpointing is enabled while your application runs.
	// Default: true.
	//
	// Experimental.
	CheckpointingEnabled *bool `field:"optional" json:"checkpointingEnabled" yaml:"checkpointingEnabled"`
	// The interval between checkpoints.
	// Default: - 1 minute.
	//
	// Experimental.
	CheckpointInterval awscdk.Duration `field:"optional" json:"checkpointInterval" yaml:"checkpointInterval"`
	// The log group to send log entries to.
	// Default: - CDK's default LogGroup.
	//
	// Experimental.
	LogGroup awslogs.ILogGroup `field:"optional" json:"logGroup" yaml:"logGroup"`
	// The level of log verbosity from the Flink application.
	// Default: FlinkLogLevel.INFO
	//
	// Experimental.
	LogLevel LogLevel `field:"optional" json:"logLevel" yaml:"logLevel"`
	// Describes the granularity of the CloudWatch metrics for an application.
	//
	// Use caution with Parallelism level metrics. Parallelism granularity logs
	// metrics for each parallel thread and can quickly become expensive when
	// parallelism is high (e.g. > 64).
	// Default: MetricsLevel.APPLICATION
	//
	// Experimental.
	MetricsLevel MetricsLevel `field:"optional" json:"metricsLevel" yaml:"metricsLevel"`
	// The minimum amount of time in to wait after a checkpoint finishes to start a new checkpoint.
	// Default: - 5 seconds.
	//
	// Experimental.
	MinPauseBetweenCheckpoints awscdk.Duration `field:"optional" json:"minPauseBetweenCheckpoints" yaml:"minPauseBetweenCheckpoints"`
	// The initial parallelism for the application.
	//
	// Kinesis Data Analytics can
	// stop the app, increase the parallelism, and start the app again if
	// autoScalingEnabled is true (the default value).
	// Default: 1.
	//
	// Experimental.
	Parallelism *float64 `field:"optional" json:"parallelism" yaml:"parallelism"`
	// The Flink parallelism allowed per Kinesis Processing Unit (KPU).
	// Default: 1.
	//
	// Experimental.
	ParallelismPerKpu *float64 `field:"optional" json:"parallelismPerKpu" yaml:"parallelismPerKpu"`
	// Configuration PropertyGroups.
	//
	// You can use these property groups to pass
	// arbitrary runtime configuration values to your Flink app.
	// Default: - No property group configuration provided to the Flink app.
	//
	// Experimental.
	PropertyGroups *map[string]*map[string]*string `field:"optional" json:"propertyGroups" yaml:"propertyGroups"`
	// Provide a RemovalPolicy to override the default.
	// Default: RemovalPolicy.DESTROY
	//
	// Experimental.
	RemovalPolicy awscdk.RemovalPolicy `field:"optional" json:"removalPolicy" yaml:"removalPolicy"`
	// A role to use to grant permissions to your application.
	//
	// Prefer omitting
	// this property and using the default role.
	// Default: - a new Role will be created.
	//
	// Experimental.
	Role awsiam.IRole `field:"optional" json:"role" yaml:"role"`
	// Security groups to use with a provided VPC.
	// Default: - a new security group is created for this application.
	//
	// Experimental.
	SecurityGroups *[]awsec2.ISecurityGroup `field:"optional" json:"securityGroups" yaml:"securityGroups"`
	// Determines if Flink snapshots are enabled.
	// Default: true.
	//
	// Experimental.
	SnapshotsEnabled *bool `field:"optional" json:"snapshotsEnabled" yaml:"snapshotsEnabled"`
	// Deploy the Flink application in a VPC.
	// Default: - no VPC.
	//
	// Experimental.
	Vpc awsec2.IVpc `field:"optional" json:"vpc" yaml:"vpc"`
	// Choose which VPC subnets to use.
	// Default: - SubnetType.PRIVATE_WITH_EGRESS subnets
	//
	// Experimental.
	VpcSubnets *awsec2.SubnetSelection `field:"optional" json:"vpcSubnets" yaml:"vpcSubnets"`
}

Props for creating an Application construct.

Example: 

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	PropertyGroups: map[string]map[string]*string{
		"FlinkApplicationProperties": map[string]*string{
			"inputStreamName": jsii.String("my-input-kinesis-stream"),
			"outputStreamName": jsii.String("my-output-kinesis-stream"),
		},
	},
	// ...
	Runtime: flink.Runtime_FLINK_1_20(),
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
})

Experimental.

type IApplication 

type IApplication interface {
	awsec2.IConnectable
	awsiam.IGrantable
	awscdk.IResource
	// Convenience method for adding a policy statement to the application role.
	// Experimental.
	AddToRolePolicy(policyStatement awsiam.PolicyStatement) *bool
	// Return a CloudWatch metric associated with this Flink application.
	// Experimental.
	Metric(metricName *string, props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time (in milliseconds) this task or operator is back pressured per second.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricBackPressuredTimeMsPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time (in milliseconds) this task or operator is busy (neither idle nor back pressured) per second.
	//
	// Can be NaN, if the value could not be
	// calculated.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricBusyTimePerMsPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The overall percentage of CPU utilization across task managers.
	//
	// For
	// example, if there are five task managers, Kinesis Data Analytics publishes
	// five samples of this metric per reporting interval.
	//
	// Units: Percentage
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricCpuUtilization(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The last watermark this application/operator/task/thread has received.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricCurrentInputWatermark(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The last watermark this application/operator/task/thread has received.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricCurrentOutputWatermark(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time elapsed during an outage for failing/recovering jobs.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricDowntime(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of times this job has fully restarted since it was submitted.
	//
	// This metric does not measure fine-grained restarts.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricFullRestarts(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Overall heap memory utilization across task managers.
	//
	// For example, if there
	// are five task managers, Kinesis Data Analytics publishes five samples of
	// this metric per reporting interval.
	//
	// Units: Percentage
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricHeapMemoryUtilization(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time (in milliseconds) this task or operator is idle (has no data to process) per second.
	//
	// Idle time excludes back pressured time, so if the task
	// is back pressured it is not idle.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricIdleTimeMsPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The number of Kinesis Processing Units that are used to run your stream processing application.
	//
	// The average number of KPUs used each hour
	// determines the billing for your application.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricKpus(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time it took to complete the last checkpoint.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricLastCheckpointDuration(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total size of the last checkpoint.
	//
	// Units: Bytes
	//
	// Reporting Level: Application.
	// Default: - maximum over 5 minutes.
	//
	// Experimental.
	MetricLastCheckpointSize(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total amount of managed memory.
	//
	// Units: Bytes
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricManagedMemoryTotal(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The amount of managed memory currently used.
	//
	// Units: Bytes
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricManagedMemoryUsed(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// Derived from managedMemoryUsed/managedMemoryTotal.
	//
	// Units: Percentage
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricManagedMemoryUtilization(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The number of times checkpointing has failed.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricNumberOfFailedCheckpoints(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The number of records this operator or task has dropped due to arriving late.
	//
	// Units: Count
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricNumLateRecordsDropped(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator, or task has received.
	//
	// Units: Count
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsIn(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator or task has received per second.
	//
	// Units: Count/Second
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsInPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator or task has emitted.
	//
	// Units: Count
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsOut(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of records this application, operator or task has emitted per second.
	//
	// Units: Count/Second
	//
	// Reporting Level: Application, Operator, Task, Parallelism.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricNumRecordsOutPerSecond(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of old garbage collection operations that have occurred across all task managers.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricOldGenerationGCCount(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total time spent performing old garbage collection operations.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - sum over 5 minutes.
	//
	// Experimental.
	MetricOldGenerationGCTime(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The total number of live threads used by the application.
	//
	// Units: Count
	//
	// Reporting Level: Application.
	// Default: - average over 5 minutes.
	//
	// Experimental.
	MetricThreadsCount(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The time that the job has been running without interruption.
	//
	// Units: Milliseconds
	//
	// Reporting Level: Application.
	// Default: - sample count over 5 minutes.
	//
	// Experimental.
	MetricUptime(props *awscloudwatch.MetricOptions) awscloudwatch.Metric
	// The application ARN.
	// Experimental.
	ApplicationArn() *string
	// The name of the Flink application.
	// Experimental.
	ApplicationName() *string
	// The application IAM role.
	// Experimental.
	Role() awsiam.IRole
}

An interface expressing the public properties on both an imported and CDK-created Flink application. Experimental.

func Application_FromApplicationArn 

func Application_FromApplicationArn(scope constructs.Construct, id *string, applicationArn *string) IApplication

Import an existing application defined outside of CDK code by applicationArn. Experimental.

func Application_FromApplicationAttributes 

func Application_FromApplicationAttributes(scope constructs.Construct, id *string, attrs *ApplicationAttributes) IApplication

Import an existing application defined outside of CDK code. Experimental.

func Application_FromApplicationName 

func Application_FromApplicationName(scope constructs.Construct, id *string, applicationName *string) IApplication

Import an existing Flink application defined outside of CDK code by applicationName. Experimental.

type LogLevel 

type LogLevel string

Available log levels for Flink applications.

Example: 

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
	Runtime: flink.Runtime_FLINK_1_20(),
	CheckpointingEnabled: jsii.Boolean(true),
	 // default is true
	CheckpointInterval: awscdk.Duration_Seconds(jsii.Number(30)),
	 // default is 1 minute
	MinPauseBetweenCheckpoints: awscdk.Duration_*Seconds(jsii.Number(10)),
	 // default is 5 seconds
	LogLevel: flink.LogLevel_ERROR,
	 // default is INFO
	MetricsLevel: flink.MetricsLevel_PARALLELISM,
	 // default is APPLICATION
	AutoScalingEnabled: jsii.Boolean(false),
	 // default is true
	Parallelism: jsii.Number(32),
	 // default is 1
	ParallelismPerKpu: jsii.Number(2),
	 // default is 1
	SnapshotsEnabled: jsii.Boolean(false),
	 // default is true
	LogGroup: logs.NewLogGroup(this, jsii.String("LogGroup")),
})

Experimental. 

const (
	// Debug level logging.
	// Experimental.
	LogLevel_DEBUG LogLevel = "DEBUG"
	// Info level logging.
	// Experimental.
	LogLevel_INFO LogLevel = "INFO"
	// Warn level logging.
	// Experimental.
	LogLevel_WARN LogLevel = "WARN"
	// Error level logging.
	// Experimental.
	LogLevel_ERROR LogLevel = "ERROR"
)

type MetricsLevel 

type MetricsLevel string

Granularity of metrics sent to CloudWatch.

Example: 

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
	Runtime: flink.Runtime_FLINK_1_20(),
	CheckpointingEnabled: jsii.Boolean(true),
	 // default is true
	CheckpointInterval: awscdk.Duration_Seconds(jsii.Number(30)),
	 // default is 1 minute
	MinPauseBetweenCheckpoints: awscdk.Duration_*Seconds(jsii.Number(10)),
	 // default is 5 seconds
	LogLevel: flink.LogLevel_ERROR,
	 // default is INFO
	MetricsLevel: flink.MetricsLevel_PARALLELISM,
	 // default is APPLICATION
	AutoScalingEnabled: jsii.Boolean(false),
	 // default is true
	Parallelism: jsii.Number(32),
	 // default is 1
	ParallelismPerKpu: jsii.Number(2),
	 // default is 1
	SnapshotsEnabled: jsii.Boolean(false),
	 // default is true
	LogGroup: logs.NewLogGroup(this, jsii.String("LogGroup")),
})

Experimental. 

const (
	// Application sends the least metrics to CloudWatch.
	// Experimental.
	MetricsLevel_APPLICATION MetricsLevel = "APPLICATION"
	// Task includes task-level metrics sent to CloudWatch.
	// Experimental.
	MetricsLevel_TASK MetricsLevel = "TASK"
	// Operator includes task-level and operator-level metrics sent to CloudWatch.
	// Experimental.
	MetricsLevel_OPERATOR MetricsLevel = "OPERATOR"
	// Send all metrics including metrics per task thread.
	// Experimental.
	MetricsLevel_PARALLELISM MetricsLevel = "PARALLELISM"
)

type PropertyGroups deprecated 

type PropertyGroups struct {
}

Interface for building AWS::KinesisAnalyticsV2::Application PropertyGroup configuration.

Example: 

// The code below shows an example of how to instantiate this type.
// The values are placeholders you should change.
import kinesisanalytics_flink_alpha "github.com/aws/aws-cdk-go/awscdkkinesisanalyticsflinkalpha"

propertyGroups := &PropertyGroups{
}

Deprecated: Use raw property bags instead (object literals, `Map<String,Object>`, etc... )

type Runtime 

type Runtime interface {
	// The Cfn string that represents a version of Flink.
	// Experimental.
	Value() *string
}

Available Flink runtimes for Kinesis Analytics.

Example: 

var bucket bucket

flinkApp := flink.NewApplication(this, jsii.String("Application"), &ApplicationProps{
	Code: flink.ApplicationCode_FromBucket(bucket, jsii.String("my-app.jar")),
	Runtime: flink.Runtime_FLINK_1_20(),
	CheckpointingEnabled: jsii.Boolean(true),
	 // default is true
	CheckpointInterval: awscdk.Duration_Seconds(jsii.Number(30)),
	 // default is 1 minute
	MinPauseBetweenCheckpoints: awscdk.Duration_*Seconds(jsii.Number(10)),
	 // default is 5 seconds
	LogLevel: flink.LogLevel_ERROR,
	 // default is INFO
	MetricsLevel: flink.MetricsLevel_PARALLELISM,
	 // default is APPLICATION
	AutoScalingEnabled: jsii.Boolean(false),
	 // default is true
	Parallelism: jsii.Number(32),
	 // default is 1
	ParallelismPerKpu: jsii.Number(2),
	 // default is 1
	SnapshotsEnabled: jsii.Boolean(false),
	 // default is true
	LogGroup: logs.NewLogGroup(this, jsii.String("LogGroup")),
})

Experimental. 

func Runtime_FLINK_1_11() Runtime
func Runtime_FLINK_1_13() Runtime
func Runtime_FLINK_1_15() Runtime
func Runtime_FLINK_1_18() Runtime
func Runtime_FLINK_1_19() Runtime
func Runtime_FLINK_1_20() Runtime
func Runtime_FLINK_1_6() Runtime
func Runtime_FLINK_1_8() Runtime

func Runtime_Of 

func Runtime_Of(value *string) Runtime

Create a new Runtime with with an arbitrary Flink version string. Experimental.

func Runtime_SQL_1_0 

func Runtime_SQL_1_0() Runtime
func Runtime_ZEPPELIN_FLINK_1_0() Runtime
func Runtime_ZEPPELIN_FLINK_2_0() Runtime
func Runtime_ZEPPELIN_FLINK_3_0() Runtime

Source Files

View all Source files

Directories

Path Synopsis
Package jsii contains the functionaility needed for jsii packages to initialize their dependencies and themselves.
 Package jsii contains the functionaility needed for jsii packages to initialize their dependencies and themselves.

Jump to

Keyboard shortcuts

? : This menu
/ : Search site
f or F : Jump to
y or Y : Canonical URL
go.dev uses cookies from Google to deliver and enhance the quality of its services and to analyze traffic. Learn more.