Init: mediaserver

This commit is contained in:
2023-02-08 12:13:28 +01:00
parent 848bc9739c
commit f7c23d4ba9
31914 changed files with 6175775 additions and 0 deletions

View File

@@ -0,0 +1,897 @@
========================
amazon.aws Release Notes
========================
.. contents:: Topics
v5.2.0
======
Release Summary
---------------
A minor release containing bugfixes for the ``ec2_eni_info`` module and the ``aws_rds`` inventory plugin, as well as improvements to the ``rds_instance`` module.
Minor Changes
-------------
- amazon.aws collection - refacterization of code to use argument specification ``fallback`` when falling back to environment variables for security credentials and AWS connection details (https://github.com/ansible-collections/amazon.aws/pull/1174).
- rds_instance - Split up the integration test-suite in a series of smaller tests (https://github.com/ansible-collections/amazon.aws/pull/1185).
- rds_instance - add support for gp3 storage type (https://github.com/ansible-collections/amazon.aws/pull/1266).
Bugfixes
--------
- aws_rds - fixes bug in RDS inventory plugin where config file was ignored (https://github.com/ansible-collections/amazon.aws/issues/1304).
- lambda - fix flaky integration test which assumes there are no other lambdas in the account (https://github.com/ansible-collections/amazon.aws/issues/1277)
v5.1.0
======
Release Summary
---------------
This release brings some minor changes, bugfixes, security fixes and deprecated features.
Minor Changes
-------------
- amazon.aws collection - The ``aws_access_key`` parameter has been renamed to ``access_key``, ``access_key`` was previously an alias for this parameter and ``aws_access_key`` remains as an alias. This change should have no observable effect for users outside the module/plugin documentation. (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``aws_secret_key`` parameter has been renamed to ``secret_key``, ``secret_key`` was previously an alias for this parameter and ``aws_secret_key`` remains as an alias. This change should have no observable effect for users outside the module/plugin documentation. (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``security_token`` parameter has been renamed to ``session_token``, ``security_token`` was previously an alias for this parameter and ``security_token`` remains as an alias. This change should have no observable effect for users outside the module/plugin documentation. (https://github.com/ansible-collections/amazon.aws/pull/1172).
- aws_account_attribute lookup plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152).
- aws_ec2 inventory - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/1181).
- aws_ec2 inventory plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152).
- aws_rds inventory plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152).
- aws_secret lookup plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152).
- aws_ssm lookup plugin - use ``missing_required_lib`` for more consistent error message when boto3/botocore is not available (https://github.com/ansible-collections/amazon.aws/pull/1152).
- ec2_instance - minor fix for launching an instance in specified AZ when ``vpc_subnet_id`` is not provided (https://github.com/ansible-collections/amazon.aws/pull/1150).
- ec2_instance - refacter ``tower_callback`` code to handle parameter validation as part of the argument specification (https://github.com/ansible-collections/amazon.aws/pull/1199).
- ec2_instance - the ``instance_role`` parameter has been renamed to ``iam_instance_profile`` to better reflect what it is, ``instance_role`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/1151).
- ec2_instance - the ``tower_callback`` parameter has been renamed to ``aap_callback``, ``tower_callback`` remains as an alias. This change should have no observable effect for users outside the module documentation (https://github.com/ansible-collections/amazon.aws/pull/1199).
- s3_object_info - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/1181).
Deprecated Features
-------------------
- amazon.aws collection - Support for the ``EC2_ACCESS_KEY`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``access_key`` parameter or ``AWS_ACCESS_KEY_ID`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - Support for the ``EC2_REGION`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``region`` parameter or ``AWS_REGION`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - Support for the ``EC2_SECRET_KEY`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``secret_key`` parameter or ``AWS_SECRET_ACCESS_KEY`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - Support for the ``EC2_SECURITY_TOKEN`` environment variable has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` parameter or ``AWS_SESSION_TOKEN`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - Support for the ``EC2_URL`` and ``S3_URL`` environment variables has been deprecated and will be removed in a release after 2024-12-01. Please use the ``endpoint_url`` parameter or ``AWS_ENDPOINT_URL`` environment variable instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``access_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``access_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``aws_security_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``ec2_access_key`` alias for the ``access_key`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``access_key`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``ec2_region`` alias for the ``region`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``region`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``ec2_secret_key`` alias for the ``secret_key`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``secret_key`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- amazon.aws collection - The ``security_token`` alias for the ``session_token`` parameter has been deprecated and will be removed in a release after 2024-12-01. Please use the ``session_token`` name instead (https://github.com/ansible-collections/amazon.aws/pull/1172).
- ec2_security_group - support for passing nested lists to ``cidr_ip`` and ``cidr_ipv6`` has been deprecated. Nested lists can be passed through the ``flatten`` filter instead ``cidr_ip: '{{ my_cidrs | flatten }}'`` (https://github.com/ansible-collections/amazon.aws/pull/1213).
- module_utils.url - ``ansible_collections.amazon.aws.module_utils.urls`` is believed to be unused and has been deprecated and will be removed in release 7.0.0.
Security Fixes
--------------
- ec2_instance - fixes leak of password into logs when using ``tower_callback.windows=True`` and ``tower_callback.set_password`` (https://github.com/ansible-collections/amazon.aws/pull/1199).
Bugfixes
--------
- ec2_instance - fixes ``Invalid type for parameter TagSpecifications, value None`` error when tags aren't specified (https://github.com/ansible-collections/amazon.aws/issues/1148).
- module_utils.transformations - ensure that ``map_complex_type`` still returns transformed items if items exists that are not in the type_map (https://github.com/ansible-collections/amazon.aws/pull/1163).
v5.0.2
======
Bugfixes
--------
- ec2_metadata_facts - fixed ``AttributeError`` (https://github.com/ansible-collections/amazon.aws/issues/1134).
v5.0.1
======
Bugfixes
--------
- ec2_vpc_net_info - fix KeyError (https://github.com/ansible-collections/amazon.aws/pull/1109).
- ec2_vpc_net_info - remove hardcoded ``ClassicLinkEnabled`` parameter when request for ``ClassicLinkDnsSupported`` failed (https://github.com/ansible-collections/amazon.aws/pull/1109).
- s3_object - be more defensive when checking the results of ``s3.get_bucket_ownership_controls`` (https://github.com/ansible-collections/amazon.aws/issues/1115).
v5.0.0
======
Release Summary
---------------
In this release we promoted many community modules to Red Hat supported status. Those modules have been moved from the commuity.aws to amazon.aws collection. This release also brings some new features, bugfixes, breaking changes and deprecated features. The amazon.aws collection has dropped support for ``botocore<1.21.0`` and ``boto3<1.18.0``. Support for ``ansible-core<2.11`` has also been dropped.
Major Changes
-------------
- autoscaling_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.autoscaling_group``.
- autoscaling_group_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.autoscaling_group_info``.
- cloudtrail - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudtrail``.
- cloudwatch_metric_alarm - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatch_metric_alarm``.
- cloudwatchevent_rule - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchevent_rule``.
- cloudwatchlogs_log_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchlogs_log_group``.
- cloudwatchlogs_log_group_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchlogs_log_group_info``.
- cloudwatchlogs_log_group_metric_filter - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.cloudwatchlogs_log_group_metric_filter``.
- ec2_eip - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_eip``.
- ec2_eip_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_eip_info``.
- elb_application_lb - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.elb_application_lb``.
- elb_application_lb_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.elb_application_lb_info``.
- execute_lambda - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.execute_lambda``.
- iam_policy - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_policy``.
- iam_policy_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_policy_info``.
- iam_user - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_user``.
- iam_user_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.iam_user_info``.
- kms_key - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.kms_key``.
- kms_key_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.kms_key_info``.
- lambda - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda``.
- lambda_alias - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_alias``.
- lambda_event - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_event``.
- lambda_execute - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_execute``.
- lambda_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_info``.
- lambda_policy - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.lambda_policy``.
- rds_cluster - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_cluster``.
- rds_cluster_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_cluster_info``.
- rds_cluster_snapshot - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_cluster_snapshot``.
- rds_instance - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_instance``.
- rds_instance_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_instance_info``.
- rds_instance_snapshot - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_instance_snapshot``.
- rds_option_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_option_group``.
- rds_option_group_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_option_group_info``.
- rds_param_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_param_group``.
- rds_snapshot_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_snapshot_info``.
- rds_subnet_group - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.rds_subnet_group``.
- route53 - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53``.
- route53_health_check - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53_health_check``.
- route53_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53_info``.
- route53_zone - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.route53_zone``.
Minor Changes
-------------
- Ability to record and replay the API interaction of a module for testing purpose. Show case the feature with an example (https://github.com/ansible-collections/amazon.aws/pull/998).
- Remove the empty __init__.py file from the distribution, they were not required anymore (https://github.com/ansible-collections/amazon.aws/pull/1018).
- amazon.aws modules - the ``ec2_url`` parameter has been renamed to ``endpoint_url`` for consistency, ``ec2_url`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/992).
- aws_caller_info - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968).
- aws_ec2 - introduce the ``allow_duplicated_hosts`` configuration key (https://github.com/ansible-collections/amazon.aws/pull/1026).
- cloudformation - avoid catching ``Exception``, catch more specific errors instead (https://github.com/ansible-collections/amazon.aws/pull/968).
- cloudwatch_metric_alarm_info - Added a new module that describes the cloudwatch metric alarms (https://github.com/ansible-collections/amazon.aws/pull/988).
- ec2_group - The ``ec2_group`` module has been renamed to ``ec2_security_group``, ``ec2_group`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/897).
- ec2_group_info - The ``ec2_group_info`` module has been renamed to ``ec2_security_group_info``, ``ec2_group_info`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/897).
- ec2_instance - Add hibernation_options and volumes->ebs->encrypted keys to support stop-hibernate instance (https://github.com/ansible-collections/amazon.aws/pull/972).
- ec2_instance - expanded the use of the automatic retries to ``InsuffienctInstanceCapacity`` (https://github.com/ansible-collections/amazon.aws/issues/1038).
- ec2_metadata_facts - avoid catching ``Exception``, catch more specific errors instead (https://github.com/ansible-collections/amazon.aws/pull/968).
- ec2_security_group - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968).
- ec2_vpc_endpoint - avoid catching ``Exception``, catch more specific errors instead (https://github.com/ansible-collections/amazon.aws/pull/968).
- ec2_vpc_nat_gateway - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968).
- ec2_vpc_net_info - handle classic link check for shared VPCs by throwing a warning instead of an error (https://github.com/ansible-collections/amazon.aws/pull/984).
- module_utils/acm - Move to jittered backoff (https://github.com/ansible-collections/amazon.aws/pull/946).
- module_utils/elbv2 - ensures that ``ip_address_type`` is set on creation rather than re-setting it after creation (https://github.com/ansible-collections/amazon.aws/pull/940).
- module_utils/elbv2 - uses new waiters with retries for temporary failures (https://github.com/ansible-collections/amazon.aws/pull/940).
- module_utils/waf - Move to jittered backoff (https://github.com/ansible-collections/amazon.aws/pull/946).
- module_utils/waiters - Add waiters to manage eks_nodegroup module (https://github.com/ansible-collections/community.aws/pull/1415).
- s3_bucket - ``rgw`` was added as an alias for the ``ceph`` parameter for consistency with the ``s3_object`` module (https://github.com/ansible-collections/amazon.aws/pull/994).
- s3_bucket - the ``s3_url`` parameter was merged into the ``endpoint_url`` parameter, ``s3_url`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/994).
- s3_object - added the ``sig_v4`` paramater, enbling the user to opt in to signature version 4 for download/get operations. (https://github.com/ansible-collections/amazon.aws/pull/1014)
- s3_object - minor linting fixes (https://github.com/ansible-collections/amazon.aws/pull/968).
- s3_object - the ``rgw`` parameter was renamed to ``ceph`` for consistency with the ``s3_bucket`` module, ``rgw`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/994).
- s3_object - the ``s3_url`` parameter was merged into the ``endpoint_url`` parameter, ``s3_url`` remains as an alias (https://github.com/ansible-collections/amazon.aws/pull/994).
- s3_object - updated module to add support for handling file upload to a bucket with ACL disabled (https://github.com/ansible-collections/amazon.aws/pull/921).
- s3_object_info - Added a new module that describes S3 Objects (https://github.com/ansible-collections/amazon.aws/pull/977).
Breaking Changes / Porting Guide
--------------------------------
- amazon.aws collection - Support for ansible-core < 2.11 has been dropped (https://github.com/ansible-collections/amazon.aws/pull/1087).
- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.21.0`` and ``boto3<1.18.0``. Most modules will continue to work with older versions of the AWS SDK, however compatability with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/934).
- doc_fragments - remove minimum collection requirements from doc_fragments/aws.py and allow pulling those from doc_fragments/aws_boto3.py instead (https://github.com/ansible-collections/amazon.aws/pull/985).
- ec2_ami - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- ec2_ami - the parameter aliases ``DeviceName``, ``VirtualName`` and ``NoDevice`` were previously deprecated and have been removed, please use ``device_name``, ``virtual_name`` and ``no_device`` instead (https://github.com/ansible-collections/amazon.aws/pull/913).
- ec2_eni_info - the mutual exclusivity of the ``eni_id`` and ``filters`` parameters is now enforced, previously ``filters`` would be ignored if ``eni_id`` was set (https://github.com/ansible-collections/amazon.aws/pull/954).
- ec2_instance - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- ec2_key - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- ec2_vol - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- ec2_vpc_dhcp_option_info - the parameter aliases ``DhcpOptionIds`` and ``DryRun`` were previously deprecated and have been removed, please use ``dhcp_options_ids`` and ``no_device`` instead (https://github.com/ansible-collections/amazon.aws/pull/913).
- ec2_vpc_endpoint - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- ec2_vpc_net - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- ec2_vpc_route_table - the default value for ``purge_tags`` has been changed from ``False`` to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/916).
- s3_bucket - the previously deprecated alias ``S3_URL`` for the ``s3_url`` parameter has been removed. Playbooks shuold be updated to use ``s3_url`` (https://github.com/ansible-collections/amazon.aws/pull/908).
- s3_object - the previously deprecated alias ``S3_URL`` for the ``s3_url`` parameter has been removed. Playbooks should be updated to use ``s3_url`` (https://github.com/ansible-collections/amazon.aws/pull/908).
Deprecated Features
-------------------
- amazon.aws collection - due to the AWS SDKs announcing the end of support for Python less than 3.7 (https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/) support for Python less than 3.7 by this collection has been deprecated and will be removed in a release after 2023-05-31 (https://github.com/ansible-collections/amazon.aws/pull/935).
- inventory/aws_ec2 - the ``include_extra_api_calls`` is now deprecated, its value is silently ignored (https://github.com/ansible-collections/amazon.aws/pull/1097).
Bugfixes
--------
- aws_ec2 - address a regression introduced in 4.1.0 (https://github.com/ansible-collections/amazon.aws/pull/862) that cause the presnse of duplicated hosts in the inventory.
- cloudtrail - Fix key error TagList to TagsList (https://github.com/ansible-collections/amazon.aws/issues/1088).
- ec2_instance - Only show the deprecation warning for the default value of ``instance_type`` when ``count`` or ``exact_count`` are specified (https://github.com//issues/980).
- ec2_metadata_facts - fix ``'NoneType' object is not callable`` exception when using Ansible 2.13+ (https://github.com/ansible-collections/amazon.aws/issues/942).
- module_utils/botocore - fix ``object has no attribute 'fail'`` error in error handling (https://github.com/ansible-collections/amazon.aws/pull/1045).
- module_utils/elbv2 - fixes ``KeyError`` when using ``UseExistingClientSecret`` rather than ``ClientSecret`` (https://github.com/ansible-collections/amazon.aws/pull/940).
- module_utils/elbv2 - improvements to idempotency when comparing listeners (https://github.com/ansible-collections/community.aws/issues/604).
- s3_object - also use ``ignore_nonexistent_bucket`` when listing a bucket (https://github.com/ansible-collections/amazon.aws/issues/966).
New Modules
-----------
- cloudtrail_info - Gather information about trails in AWS Cloud Trail.
- cloudwatch_metric_alarm_info - Gather information about the alarms for the specified metric
- s3_object_info - Gather information about objects in S3
v4.3.0
======
Release Summary
---------------
The amazon.aws 4.3.0 release includes a number of minor bug fixes and improvements.
Following the release of amazon.aws 5.0.0, backports to the 4.x series will be limited to
security issues and bugfixes.
Minor Changes
-------------
- ec2_instance - expanded the use of the automatic retries to ``InsuffienctInstanceCapacity`` (https://github.com/ansible-collections/amazon.aws/issues/1038).
Bugfixes
--------
- ec2_metadata_facts - fix ``'NoneType' object is not callable`` exception when using Ansible 2.13+ (https://github.com/ansible-collections/amazon.aws/issues/942).
- module_utils/cloud - Fix ``ValueError: ansible_collections.amazon.aws.plugins.module_utils.core.__spec__ is None`` error on Ansible 2.9 (https://github.com/ansible-collections/amazon.aws/issues/1083).
v4.2.0
======
Minor Changes
-------------
- ec2_security_group - set type as ``list`` for rules->group_name as it can accept both ``str`` and ``list`` (https://github.com/ansible-collections/amazon.aws/pull/971).
- various modules - linting fixups (https://github.com/ansible-collections/amazon.aws/pull/953).
Deprecated Features
-------------------
- module_utils.cloud - removal of the ``CloudRetry.backoff`` has been delayed until release 6.0.0. It is recommended to update custom modules to use ``jittered_backoff`` or ``exponential_backoff`` instead (https://github.com/ansible-collections/amazon.aws/pull/951).
v4.1.0
======
Minor Changes
-------------
- ec2_instance - expanded the use of the automatic retries on temporary failures (https://github.com/ansible-collections/amazon.aws/issues/927).
- s3_bucket - updated module to enable support for setting S3 Bucket Keys for SSE-KMS (https://github.com/ansible-collections/amazon.aws/pull/882).
Deprecated Features
-------------------
- amazon.aws collection - due to the AWS SDKs announcing the end of support for Python less than 3.7 (https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/) support for Python less than 3.7 by this collection has been deprecated and will be removed in a release after 2023-05-31 (https://github.com/ansible-collections/amazon.aws/pull/935).
Bugfixes
--------
- aws_ec2 - ensure the correct number of hosts are returned when tags as hostnames are used (https://github.com/ansible-collections/amazon.aws/pull/862).
- elb_application_lb - fix ``KeyError`` when balancing across two Target Groups (https://github.com/ansible-collections/community.aws/issues/1089).
- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB in check mode with a health check (https://github.com/ansible-collections/amazon.aws/pull/915).
- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB using security group names (https://github.com/ansible-collections/amazon.aws/issues/914).
v4.0.0
======
Major Changes
-------------
- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.20.0`` and ``boto3<1.17.0``. Most modules will continue to work with older versions of the AWS SDK, however compatability with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/574).
Minor Changes
-------------
- aws_s3 - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615).
- aws_s3 - The ``aws_s3`` module has been renamed to ``s3_object`` (https://github.com/ansible-collections/amazon.aws/pull/869).
- aws_s3 - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845).
- ec2_eni - Change parameter ``device_index`` data type to string when passing to ``describe_network_inter`` api call (https://github.com/ansible-collections/amazon.aws/pull/877).
- ec2_eni - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845).
- ec2_group - add ``egress_rules`` as an alias for ``rules_egress`` (https://github.com/ansible-collections/amazon.aws/pull/878).
- ec2_group - add ``purge_egress_rules`` as an alias for ``purge_rules_egress`` (https://github.com/ansible-collections/amazon.aws/pull/878).
- ec2_instance - Add missing ``metadata_options`` parameters (https://github.com/ansible-collections/amazon.aws/pull/715).
- ec2_key - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845).
- ec2_vpc_net - add support for managing VPCs by ID (https://github.com/ansible-collections/amazon.aws/pull/848).
- ec2_vpc_subnet - add support for OutpostArn param (https://github.com/ansible-collections/amazon.aws/pull/598).
- elb_classic_lb - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845).
- s3_bucket - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615).
- s3_bucket - ``resource_tags`` has been added as an alias for the ``tags`` parameter (https://github.com/ansible-collections/amazon.aws/pull/845).
Breaking Changes / Porting Guide
--------------------------------
- Tags beginning with ``aws:`` will not be removed when purging tags, these tags are reserved by Amazon and may not be updated or deleted (https://github.com/ansible-collections/amazon.aws/issues/817).
- amazon.aws collection - the ``profile`` parameter is now mutually exclusive with the ``aws_access_key``, ``aws_secret_key`` and ``security_token`` parameters (https://github.com/ansible-collections/amazon.aws/pull/834).
- aws_az_info - the module alias ``aws_az_facts`` was deprecated in Ansible 2.9 and has now been removed (https://github.com/ansible-collections/amazon.aws/pull/832).
- aws_s3 - the default value for ``ensure overwrite`` has been changed to ``different`` instead of ``always`` so that the module is idempotent by default (https://github.com/ansible-collections/amazon.aws/issues/811).
- aws_ssm - on_denied and on_missing now both default to error, for consistency with both aws_secret and the base Lookup class (https://github.com/ansible-collections/amazon.aws/issues/617).
- ec2 - The ``ec2`` module has been removed in release 4.0.0 and replaced by the ``ec2_instance`` module (https://github.com/ansible-collections/amazon.aws/pull/630).
- ec2_vpc_igw_info - The default value for ``convert_tags`` has been changed to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/835).
- elb_classic_lb - the ``ec2_elb`` fact has been removed (https://github.com/ansible-collections/amazon.aws/pull/827).
- module_utils - Support for the original AWS SDK aka ``boto`` has been removed, including all relevant helper functions. All modules should now use the ``boto3``/``botocore`` AWS SDK (https://github.com/ansible-collections/amazon.aws/pull/630)
Deprecated Features
-------------------
- aws_s3 - The ``S3_URL`` alias for the s3_url option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- ec2_ami - The ``DeviceName`` alias for the device_name option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- ec2_ami - The ``NoDevice`` alias for the no_device option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- ec2_ami - The ``VirtualName`` alias for the virtual_name option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- ec2_ami - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846).
- ec2_instance - The default value for ```instance_type``` has been deprecated, in the future release you must set an instance_type or a launch_template (https://github.com/ansible-collections/amazon.aws/pull/587).
- ec2_instance - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/849).
- ec2_key - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846).
- ec2_vol - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846).
- ec2_vpc_dhcp_option_info - The ``DhcpOptionIds`` alias for the dhcp_option_ids option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- ec2_vpc_dhcp_option_info - The ``DryRun`` alias for the dry_run option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- ec2_vpc_endpoint - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846).
- ec2_vpc_net - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/848).
- ec2_vpc_route_table - the current default value of ``False`` for ``purge_tags`` has been deprecated and will be updated in release 5.0.0 to ``True`` (https://github.com/ansible-collections/amazon.aws/pull/846).
- s3_bucket - The ``S3_URL`` alias for the s3_url option has been deprecated and will be removed in release 5.0.0 (https://github.com/ansible-collections/community.aws/pull/795).
- s3_object - Support for creation and deletion of S3 buckets has been deprecated. Please use the ``amazon.aws.s3_bucket`` module to create and delete buckets (https://github.com/ansible-collections/amazon.aws/pull/869).
Removed Features (previously deprecated)
----------------------------------------
- cloudformation - the ``template_format`` option has been removed. It has been ignored by the module since Ansible 2.3 (https://github.com/ansible-collections/amazon.aws/pull/833).
- ec2_key - the ``wait_timeout`` option had no effect, was deprecated in release 1.0.0, and has now been removed (https://github.com/ansible-collections/amazon.aws/pull/830).
- ec2_key - the ``wait`` option had no effect, was deprecated in release 1.0.0, and has now been removed (https://github.com/ansible-collections/amazon.aws/pull/830).
- ec2_tag - the previously deprecated state ``list`` has been removed. To list tags on an EC2 resource the ``ec2_tag_info`` module can be used (https://github.com/ansible-collections/amazon.aws/pull/829).
- ec2_vol - the previously deprecated state ``list`` has been removed. To list volumes the ``ec2_vol_info`` module can be used (https://github.com/ansible-collections/amazon.aws/pull/828).
- module_utils.batch - the class ``ansible_collections.amazon.aws.plugins.module_utils.batch.AWSConnection`` has been removed. Please use ``AnsibleAWSModule.client()`` instead (https://github.com/ansible-collections/amazon.aws/pull/831).
Bugfixes
--------
- ec2_group - fix uncaught exception when running with ``--diff`` and ``--check`` to create a new security group (https://github.com/ansible-collections/amazon.aws/issues/440).
- ec2_instance - Add a condition to handle default ```instance_type``` value for fix breaking on instance creation with launch template (https://github.com/ansible-collections/amazon.aws/pull/587).
- ec2_instance - raise an error when missing permission to stop instance when ``state`` is set to ``rebooted``` (https://github.com/ansible-collections/amazon.aws/pull/671).
- ec2_vpc_igw - use gateway_id rather than filters to paginate if possible to fix 'NoneType' object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/766).
- ec2_vpc_net - fix a bug where CIDR configuration would be updated in check mode (https://github.com/ansible/ansible/issues/62678).
- ec2_vpc_net - fix a bug where the module would get stuck if DNS options were updated in check mode (https://github.com/ansible/ansible/issues/62677).
- elb_classic_lb - modify the return value of _format_listeners method to resolve a failure creating https listeners (https://github.com/ansible-collections/amazon.aws/pull/860).
v3.5.0
======
Release Summary
---------------
Following the release of amazon.aws 5.0.0, 3.5.0 is a bugfix release and the final planned release for the 3.x series.
Minor Changes
-------------
- ec2_security_group - set type as ``list`` for rules->group_name as it can accept both ``str`` and ``list`` (https://github.com/ansible-collections/amazon.aws/pull/971).
Bugfixes
--------
- ec2_metadata_facts - fix ``'NoneType' object is not callable`` exception when using Ansible 2.13+ (https://github.com/ansible-collections/amazon.aws/issues/942).
v3.4.0
======
Minor Changes
-------------
- ec2_instance - expanded the use of the automatic retries on temporary failures (https://github.com/ansible-collections/amazon.aws/issues/927).
Bugfixes
--------
- elb_application_lb - fix ``KeyError`` when balancing across two Target Groups (https://github.com/ansible-collections/community.aws/issues/1089).
- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB in check mode with a health check (https://github.com/ansible-collections/amazon.aws/pull/915).
- elb_classic_lb - fix ``'NoneType' object has no attribute`` bug when creating a new ELB using security group names (https://github.com/ansible-collections/amazon.aws/issues/914).
v3.3.1
======
v3.3.0
======
Minor Changes
-------------
- aws_ec2 inventory - Allow for literal strings in hostname that don't match filter parameters in ec2 describe-instances (https://github.com/ansible-collections/amazon.aws/pull/826).
- aws_ssm - Add support for ``endpoint`` parameter (https://github.com/ansible-collections/amazon.aws/pull/837).
- module.utils.rds - add retry_codes to get_rds_method_attribute return data to use in call_method and add unit tests (https://github.com/ansible-collections/amazon.aws/pull/776).
- module.utils.rds - refactor to utilize get_rds_method_attribute return data (https://github.com/ansible-collections/amazon.aws/pull/776).
- module_utils - add new aliases ``aws_session_token`` and ``session_token`` to the ``security_token`` parameter to be more in-line with the boto SDK (https://github.com/ansible-collections/amazon.aws/pull/631).
- module_utils.rds - Add support and unit tests for addition/removal of IAM roles to/from a db instance in module_utils.rds with waiters (https://github.com/ansible-collections/amazon.aws/pull/714).
Bugfixes
--------
- Include ``PSF-license.txt`` file for ``plugins/module_utils/_version.py``.
- aws_account_attribute lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- aws_ec2 inventory plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- aws_rds inventory plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- aws_resource_actions callback plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- aws_secret lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- aws_service_ip_ranges lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- aws_ssm - Fix environment variables for client configuration (e.g., AWS_PROFILE, AWS_ACCESS_KEY_ID) (https://github.com/ansible-collections/amazon.aws/pull/837).
- aws_ssm lookup plugin - fix linting errors in documentation data (https://github.com/ansible-collections/amazon.aws/pull/701).
- ec2_instance - ec2_instance module broken in Python 3.8 - dict keys modified during iteration (https://github.com/ansible-collections/amazon.aws/issues/709).
- module.utils.rds - Add waiter for promoting read replica to fix idempotency issue (https://github.com/ansible-collections/amazon.aws/pull/714).
- module.utils.rds - Catch InvalidDBSecurityGroupStateFault when modifying a db instance (https://github.com/ansible-collections/amazon.aws/pull/776).
- module.utils.s3 - Update validate_bucket_name minimum length to 3 (https://github.com/ansible-collections/amazon.aws/pull/802).
v3.2.0
======
Minor Changes
-------------
- aws_secret - add pagination for ``bypath`` functionality (https://github.com/ansible-collections/amazon.aws/pull/591).
- ec2_instance - Fix scope of deprecation warning to not show warning when ``state`` in ``absent`` (https://github.com/ansible-collections/amazon.aws/pull/719).
- ec2_vpc_route_table - support associating internet gateways (https://github.com/ansible-collections/amazon.aws/pull/690).
- module_utils.elbv2 - Add support for alb specific attributes and compare_elb_attributes method to support check_mode in module_utils.elbv2 (https://github.com/ansible-collections/amazon.aws/pull/696).
- s3_bucket - Add support for enforced bucket owner object ownership (https://github.com/ansible-collections/amazon.aws/pull/694).
Bugfixes
--------
- aws_ec2 inventory - use the iam_role_arn configuration parameter to assume the role before trying to call DescribeRegions if the regions configuration is not set and AWS credentials provided without enough privilege to perform the DescribeRegions action. (https://github.com/ansible-collections/amazon.aws/issues/566).
- ec2_vol - changing a volume from a type that does not support IOPS (like ``standard``) to a type that does (like ``gp3``) fails (https://github.com/ansible-collections/amazon.aws/issues/626).
- ec2_vpc_igw - fix 'NoneType' object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/691).
- ec2_vpc_igw - use paginator for describe internet gateways and add retry to fix NoneType object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/695).
- ec2_vpc_net - In check mode, ensure the module does not change the configuration. Handle case when Amazon-provided ipv6 block is enabled, then disabled, then enabled again. Do not disable IPv6 CIDR association (using Amazon pool) if ipv6_cidr property is not present in the task. If the VPC already exists and ipv6_cidr property, retain the current config (https://github.com/ansible-collections/amazon.aws/pull/631).
v3.1.1
======
Minor Changes
-------------
- bump the release version of the amazon.aws collection from 3.1.0 to 3.1.1 because of a bug that occurred while uploading to Galaxy.
v3.1.0
======
Minor Changes
-------------
- add new parameters hostvars_prefix and hostvars_suffix for inventory plugins aws_ec2 and aws_rds (https://github.com/ansible-collections/amazon.aws/issues/535).
- aws_s3 - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615).
- aws_s3 - add latest choice on ``overwrite`` parameter to get latest object on S3 (https://github.com/ansible-collections/amazon.aws/pull/595).
- ec2_vol - add support for OutpostArn param (https://github.com/ansible-collections/amazon.aws/pull/597).
- ec2_vol - tag volume on creation (https://github.com/ansible-collections/amazon.aws/pull/603).
- ec2_vpc_route_table - add support for IPv6 in creating route tables (https://github.com/ansible-collections/amazon.aws/pull/601).
- s3_bucket - Add ``validate_bucket_name`` option, to control bucket name validation (https://github.com/ansible-collections/amazon.aws/pull/615).
Deprecated Features
-------------------
- ec2_instance - The default value for ```instance_type``` has been deprecated, in the future release you must set an instance_type or a launch_template (https://github.com/ansible-collections/amazon.aws/pull/587).
Bugfixes
--------
- Various modules and plugins - use vendored version of ``distutils.version`` instead of the deprecated Python standard library ``distutils`` (https://github.com/ansible-collections/amazon.aws/pull/599).
- aws_acm - No longer raising ResourceNotFound exception while retrieving ACM certificates.
- aws_s3 - fix exception raised when using module to copy from source to destination and key is missing from source (https://github.com/ansible-collections/amazon.aws/issues/602).
- ec2_instance - Add a condition to handle default ```instance_type``` value for fix breaking on instance creation with launch template (https://github.com/ansible-collections/amazon.aws/pull/587).
- ec2_key - add support for ED25519 key type (https://github.com/ansible-collections/amazon.aws/issues/572).
- ec2_vol - Sets the Iops value in req_obj even if the iops value has not changed, to allow modifying volume types that require passing an iops value to boto. (https://github.com/ansible-collections/amazon.aws/pull/606)
- elb_classic_lb - handle security_group_ids when providing security_group_names and fix broken tasks in integration test (https://github.com/ansible-collections/amazon.aws/pull/592).
- s3_bucket - Enable the management of bucket-level ACLs (https://github.com/ansible-collections/amazon.aws/issues/573).
v3.0.0
======
Major Changes
-------------
- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.19.0`` and ``boto3<1.16.0``. Most modules will continue to work with older versions of the AWS SDK, however compatability with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/574).
Minor Changes
-------------
- ec2_instance - add count parameter support (https://github.com/ansible-collections/amazon.aws/pull/539).
Breaking Changes / Porting Guide
--------------------------------
- aws_caller_facts - Remove deprecated ``aws_caller_facts`` alias. Please use ``aws_caller_info`` instead.
- cloudformation_facts - Remove deprecated ``cloudformation_facts`` alias. Please use ``cloudformation_info`` instead.
- ec2_ami_facts - Remove deprecated ``ec2_ami_facts`` alias. Please use ``ec2_ami_info`` instead.
- ec2_eni_facts - Remove deprecated ``ec2_eni_facts`` alias. Please use ``ec2_eni_info`` instead.
- ec2_group_facts - Remove deprecated ``ec2_group_facts`` alias. Please use ``ec2_group_info`` instead.
- ec2_instance_facts - Remove deprecated ``ec2_instance_facts`` alias. Please use ``ec2_instance_info`` instead.
- ec2_snapshot_facts - Remove deprecated ``ec2_snapshot_facts`` alias. Please use ``ec2_snapshot_info`` instead.
- ec2_vol_facts - Remove deprecated ``ec2_vol_facts`` alias. Please use ``ec2_vol_info`` instead.
- ec2_vpc_dhcp_option_facts - Remove deprecated ``ec2_vpc_dhcp_option_facts`` alias. Please use ``ec2_vpc_dhcp_option_info`` instead.
- ec2_vpc_endpoint_facts - Remove deprecated ``ec2_vpc_endpoint_facts`` alias. Please use ``ec2_vpc_endpoint_info`` instead.
- ec2_vpc_igw_facts - Remove deprecated ``ec2_vpc_igw_facts`` alias. Please use ``ec2_vpc_igw_info`` instead.
- ec2_vpc_nat_gateway_facts - Remove deprecated ``ec2_vpc_nat_gateway_facts`` alias. Please use ``ec2_vpc_nat_gateway_info`` instead.
- ec2_vpc_net_facts - Remove deprecated ``ec2_vpc_net_facts`` alias. Please use ``ec2_vpc_net_info`` instead.
- ec2_vpc_route_table_facts - Remove deprecated ``ec2_vpc_route_table_facts`` alias. Please use ``ec2_vpc_route_table_info`` instead.
- ec2_vpc_subnet_facts - Remove deprecated ``ec2_vpc_subnet_facts`` alias. Please use ``ec2_vpc_subnet_info`` instead.
Deprecated Features
-------------------
- module_utils - support for the original AWS SDK ``boto`` has been deprecated in favour of the ``boto3``/``botocore`` SDK. All ``boto`` based modules have either been deprecated or migrated to ``botocore``, and the remaining support code in module_utils will be removed in release 4.0.0 of the amazon.aws collection. Any modules outside of the amazon.aws and community.aws collections based on the ``boto`` library will need to be migrated to the ``boto3``/``botocore`` libraries (https://github.com/ansible-collections/amazon.aws/pull/575).
v2.2.0
======
Minor Changes
-------------
- ec2_instance - add count parameter support (https://github.com/ansible-collections/amazon.aws/pull/539).
Bugfixes
--------
- aws_ec2 inventory - use the iam_role_arn configuration parameter to assume the role before trying to call DescribeRegions if the regions configuration is not set and AWS credentials provided without enough privilege to perform the DescribeRegions action. (https://github.com/ansible-collections/amazon.aws/issues/566).
- ec2_vol - Sets the Iops value in req_obj even if the iops value has not changed, to allow modifying volume types that require passing an iops value to boto. (https://github.com/ansible-collections/amazon.aws/pull/606)
- ec2_vol - changing a volume from a type that does not support IOPS (like ``standard``) to a type that does (like ``gp3``) fails (https://github.com/ansible-collections/amazon.aws/issues/626).
- ec2_vpc_igw - fix 'NoneType' object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/691).
- ec2_vpc_igw - use paginator for describe internet gateways and add retry to fix NoneType object is not subscriptable error (https://github.com/ansible-collections/amazon.aws/pull/695).
- elb_classic_lb - handle security_group_ids when providing security_group_names and fix broken tasks in integration test (https://github.com/ansible-collections/amazon.aws/pull/592).
v2.1.0
======
Minor Changes
-------------
- aws_service_ip_ranges - add new option ``ipv6_prefixes`` to get only IPV6 addresses and prefixes for Amazon services (https://github.com/ansible-collections/amazon.aws/pull/430)
- cloudformation - fix detection when there are no changes. Sometimes when there are no changes, the change set will have a status FAILED with StatusReason No updates are to be performed (https://github.com/ansible-collections/amazon.aws/pull/507).
- ec2_ami - add check_mode support (https://github.com/ansible-collections/amazon.aws/pull/516).
- ec2_ami - use module_util helper for tagging AMIs (https://github.com/ansible-collections/amazon.aws/pull/520).
- ec2_ami - when creating an AMI from an instance pass the tagging options at creation time (https://github.com/ansible-collections/amazon.aws/pull/551).
- ec2_elb_lb - module renamed to ``elb_classic_lb`` (https://github.com/ansible-collections/amazon.aws/pull/377).
- ec2_eni - add check mode support (https://github.com/ansible-collections/amazon.aws/pull/534).
- ec2_eni - use module_util helper for tagging ENIs (https://github.com/ansible-collections/amazon.aws/pull/522).
- ec2_instance - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/527).
- ec2_key - add support for tagging key pairs (https://github.com/ansible-collections/amazon.aws/pull/548).
- ec2_snapshot - add check_mode support (https://github.com/ansible-collections/amazon.aws/pull/512).
- ec2_vol - add check_mode support (https://github.com/ansible-collections/amazon.aws/pull/509).
- ec2_vpc_dhcp_option - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531).
- ec2_vpc_endpoint - added ``vpc_endpoint_security_groups`` parameter to support defining the security group attached to an interface endpoint (https://github.com/ansible-collections/amazon.aws/pull/544).
- ec2_vpc_endpoint - added ``vpc_endpoint_subnets`` parameter to support defining the subnet attached to an interface or gateway endpoint (https://github.com/ansible-collections/amazon.aws/pull/544).
- ec2_vpc_endpoint - use module_util helper for tagging (https://github.com/ansible-collections/amazon.aws/pull/525).
- ec2_vpc_endpoint - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531).
- ec2_vpc_igw - use module_util helper for tagging (https://github.com/ansible-collections/amazon.aws/pull/523).
- ec2_vpc_igw - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531).
- ec2_vpc_nat_gateway - use module_util helper for tagging (https://github.com/ansible-collections/amazon.aws/pull/524).
- ec2_vpc_nat_gateway - use module_util helpers for tagging (https://github.com/ansible-collections/amazon.aws/pull/531).
- elb_classic_lb - added retries on common AWS temporary API failures (https://github.com/ansible-collections/amazon.aws/pull/377).
- elb_classic_lb - added support for check_mode (https://github.com/ansible-collections/amazon.aws/pull/377).
- elb_classic_lb - added support for wait during creation (https://github.com/ansible-collections/amazon.aws/pull/377).
- elb_classic_lb - added support for wait during instance addition and removal (https://github.com/ansible-collections/amazon.aws/pull/377).
- elb_classic_lb - migrated to boto3 SDK (https://github.com/ansible-collections/amazon.aws/pull/377).
- elb_classic_lb - various error messages changed due to refactor (https://github.com/ansible-collections/amazon.aws/pull/377).
- module_utils.ec2 - moved generic tagging helpers into module_utils.tagging (https://github.com/ansible-collections/amazon.aws/pull/527).
- module_utils.tagging - add new helper to generate TagSpecification lists (https://github.com/ansible-collections/amazon.aws/pull/527).
Deprecated Features
-------------------
- ec2_classic_lb - setting of the ``ec2_elb`` fact has been deprecated and will be removed in release 4.0.0 of the collection. The module now returns ``elb`` which can be accessed using the register keyword (https://github.com/ansible-collections/amazon.aws/pull/552).
Bugfixes
--------
- AWS action group - added missing ``ec2_instance_facts`` entry (https://github.com/ansible-collections/amazon.aws/issues/557)
- ec2_ami - fix problem when creating an AMI from an instance with ephemeral volumes (https://github.com/ansible-collections/amazon.aws/issues/511).
- ec2_instance - ensure that ec2_instance falls back to the tag(Name) parameter when no filter and no name parameter is passed (https://github.com/ansible-collections/amazon.aws/issues/526).
- s3_bucket - update error handling to better support DigitalOcean Space (https://github.com/ansible-collections/amazon.aws/issues/508).
v2.0.0
======
Major Changes
-------------
- amazon.aws collection - Due to the AWS SDKs announcing the end of support for Python less than 3.6 (https://boto3.amazonaws.com/v1/documentation/api/1.17.64/guide/migrationpy3.html) this collection now requires Python 3.6+ (https://github.com/ansible-collections/amazon.aws/pull/298).
- amazon.aws collection - The amazon.aws collection has dropped support for ``botocore<1.18.0`` and ``boto3<1.15.0``. Most modules will continue to work with older versions of the AWS SDK, however compatability with older versions of the SDK is not guaranteed and will not be tested. When using older versions of the SDK a warning will be emitted by Ansible (https://github.com/ansible-collections/amazon.aws/pull/502).
- ec2_instance - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_instance``.
- ec2_instance_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_instance_info``.
- ec2_vpc_endpoint - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint``.
- ec2_vpc_endpoint_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint_info``.
- ec2_vpc_endpoint_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint_info``.
- ec2_vpc_endpoint_service_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_endpoint_service_info``.
- ec2_vpc_igw - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_igw``.
- ec2_vpc_igw_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_igw_facts``.
- ec2_vpc_igw_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_igw_info``.
- ec2_vpc_nat_gateway - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_nat_gateway``.
- ec2_vpc_nat_gateway_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_nat_gateway_info``.
- ec2_vpc_nat_gateway_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_nat_gateway_info``.
- ec2_vpc_route_table - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_route_table``.
- ec2_vpc_route_table_facts - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_route_table_facts``.
- ec2_vpc_route_table_info - The module has been migrated from the ``community.aws`` collection. Playbooks using the Fully Qualified Collection Name for this module should be updated to use ``amazon.aws.ec2_vpc_route_table_info``.
Minor Changes
-------------
- aws_ec2 - use a generator rather than list comprehension (https://github.com/ansible-collections/amazon.aws/pull/465).
- aws_s3 - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- aws_s3 - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- aws_s3 - add ``tags`` and ``purge_tags`` features for an S3 object (https://github.com/ansible-collections/amazon.aws/pull/335)
- aws_s3 - new mode to copy existing on another bucket (https://github.com/ansible-collections/amazon.aws/pull/359).
- aws_secret - added support for gracefully handling deleted secrets (https://github.com/ansible-collections/amazon.aws/pull/455).
- aws_ssm - add "on_missing" and "on_denied" option (https://github.com/ansible-collections/amazon.aws/pull/370).
- cloudformation - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- cloudformation - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_ami - ensure tags are propagated to the snapshot(s) when creating an AMI (https://github.com/ansible-collections/amazon.aws/pull/437).
- ec2_eni - fix idempotency when ``security_groups`` attribute is specified (https://github.com/ansible-collections/amazon.aws/pull/337).
- ec2_eni - timeout increased when waiting for ENIs to finish detaching (https://github.com/ansible-collections/amazon.aws/pull/501).
- ec2_group - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_group - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_group - use a generator rather than list comprehension (https://github.com/ansible-collections/amazon.aws/pull/465).
- ec2_group - use system ipaddress module, available with Python >= 3.3, instead of vendored copy (https://github.com/ansible-collections/amazon.aws/pull/461).
- ec2_instance - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_instance - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_instance - add ``throughput`` parameter for gp3 volume types (https://github.com/ansible-collections/amazon.aws/pull/433).
- ec2_instance - add support for controlling metadata options (https://github.com/ansible-collections/amazon.aws/pull/414).
- ec2_instance - remove unnecessary raise when exiting with a failure (https://github.com/ansible-collections/amazon.aws/pull/460).
- ec2_instance_info - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_instance_info - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_snapshot - migrated to use the boto3 python library (https://github.com/ansible-collections/amazon.aws/pull/356).
- ec2_spot_instance_info - Added a new module that describes the specified Spot Instance requests (https://github.com/ansible-collections/amazon.aws/pull/487).
- ec2_vol - add parameter ``multi_attach`` to support Multi-Attach on volume creation/update (https://github.com/ansible-collections/amazon.aws/pull/362).
- ec2_vol - relax the boto3/botocore requirements and only require botocore 1.19.27 for modifying the ``throughput`` parameter (https://github.com/ansible-collections/amazon.aws/pull/346).
- ec2_vpc_dhcp_option - Now also returns a boto3-style resource description in the ``dhcp_options`` result key. This includes any tags for the ``dhcp_options_id`` and has the same format as the current return value of ``ec2_vpc_dhcp_option_info``. (https://github.com/ansible-collections/amazon.aws/pull/252)
- ec2_vpc_dhcp_option_info - Now also returns a user-friendly ``dhcp_config`` key that matches the historical ``new_config`` key from ec2_vpc_dhcp_option, and alleviates the need to use ``items2dict(key_name='key', value_name='values')`` when parsing the output of the module. (https://github.com/ansible-collections/amazon.aws/pull/252)
- ec2_vpc_subnet - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- ec2_vpc_subnet - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- integration tests - remove dependency with collection ``community.general`` (https://github.com/ansible-collections/amazon.aws/pull/361).
- module_utils/waiter - add RDS cluster ``cluster_available`` waiter (https://github.com/ansible-collections/amazon.aws/pull/464).
- module_utils/waiter - add RDS cluster ``cluster_deleted`` waiter (https://github.com/ansible-collections/amazon.aws/pull/464).
- module_utils/waiter - add Route53 ``resource_record_sets_changed`` waiter (https://github.com/ansible-collections/amazon.aws/pull/350).
- s3_bucket - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- s3_bucket - Tests for compatability with older versions of the AWS SDKs have been removed (https://github.com/ansible-collections/amazon.aws/pull/442).
- s3_bucket - add new option ``object_ownership`` to configure object ownership (https://github.com/ansible-collections/amazon.aws/pull/311)
- s3_bucket - updated to use HeadBucket instead of ListBucket when testing for bucket existence (https://github.com/ansible-collections/amazon.aws/pull/357).
Breaking Changes / Porting Guide
--------------------------------
- ec2_instance - instance wait for state behaviour has changed. If plays require the old behavior of waiting for the instance monitoring status to become ``OK`` when launching a new instance, the action will need to specify ``state: started`` (https://github.com/ansible-collections/amazon.aws/pull/481).
- ec2_snapshot - support for waiting indefinitely has been dropped, new default is 10 minutes (https://github.com/ansible-collections/amazon.aws/pull/356).
- ec2_vol_info - return ``attachment_set`` is now a list of attachments with Multi-Attach support on disk. (https://github.com/ansible-collections/amazon.aws/pull/362).
- ec2_vpc_dhcp_option - The module has been refactored to use boto3. Keys and value types returned by the module are now consistent, which is a change from the previous behaviour. A ``purge_tags`` option has been added, which defaults to ``True``. (https://github.com/ansible-collections/amazon.aws/pull/252)
- ec2_vpc_dhcp_option_info - Now preserves case for tag keys in return value. (https://github.com/ansible-collections/amazon.aws/pull/252)
- module_utils.core - The boto3 switch has been removed from the region parameter (https://github.com/ansible-collections/amazon.aws/pull/287).
- module_utils/compat - vendored copy of ipaddress removed (https://github.com/ansible-collections/amazon.aws/pull/461).
- module_utils/core - updated the ``scrub_none_parameters`` function so that ``descend_into_lists`` is set to ``True`` by default (https://github.com/ansible-collections/amazon.aws/pull/297).
Deprecated Features
-------------------
- ec2 - the boto based ``ec2`` module has been deprecated in favour of the boto3 based ``ec2_instance`` module. The ``ec2`` module will be removed in release 4.0.0 (https://github.com/ansible-collections/amazon.aws/pull/424).
- ec2_vpc_dhcp_option - The ``new_config`` return key has been deprecated and will be removed in a future release. It will be replaced by ``dhcp_config``. Both values are returned in the interim. (https://github.com/ansible-collections/amazon.aws/pull/252)
Bugfixes
--------
- aws_s3 - Fix upload permission when an S3 bucket ACL policy requires a particular canned ACL (https://github.com/ansible-collections/amazon.aws/pull/318)
- ec2_ami - Fix ami issue when creating an ami with no_device parameter (https://github.com/ansible-collections/amazon.aws/pull/386)
- ec2_instance - ``ec2_instance`` was waiting on EC2 instance monitoring status to be ``OK`` when launching a new instance. This could cause a play to wait multiple minutes for AWS's monitoring to complete status checks (https://github.com/ansible-collections/amazon.aws/pull/481).
- ec2_snapshot - Fix snapshot issue when capturing a snapshot of a volume without tags (https://github.com/ansible-collections/amazon.aws/pull/383)
- ec2_vol - Fixes ``changed`` status when ``modify_volume`` is used, but no new disk is being attached. The module incorrectly reported that no change had occurred even when disks had been modified (iops, throughput, type, etc.). (https://github.com/ansible-collections/amazon.aws/issues/482).
- ec2_vol - fix iops setting and enforce iops/throughput parameters usage (https://github.com/ansible-collections/amazon.aws/pull/334)
- inventory - ``include_filters`` won't be ignored anymore if ``filters`` is not set (https://github.com/ansible-collections/amazon.aws/issues/457).
- s3_bucket - Fix error handling when attempting to set a feature that is not implemented (https://github.com/ansible-collections/amazon.aws/pull/391).
- s3_bucket - Gracefully handle ``NotImplemented`` exceptions when fetching encryption settings (https://github.com/ansible-collections/amazon.aws/issues/390).
New Modules
-----------
- ec2_spot_instance - request, stop, reboot or cancel spot instance
- ec2_spot_instance_info - Gather information about ec2 spot instance requests
v1.5.0
======
Minor Changes
-------------
- AWS inventory plugins - use shared HAS_BOTO3 helper rather than copying code (https://github.com/ansible-collections/amazon.aws/pull/288).
- AWS lookup plugins - use shared HAS_BOTO3 helper rather than copying code (https://github.com/ansible-collections/amazon.aws/pull/288).
- aws_account_attribute - add retries on common AWS failures (https://github.com/ansible-collections/amazon.aws/pull/295).
- aws_ec2 inventory - expose a new configuration key ``use_contrib_script_compatible_ec2_tag_keys`` to reproduce a behavior of the old ``ec2.py`` inventory script. With this option enabled, each tag is exposed using a ``ec2_tag_TAGNAME`` key (https://github.com/ansible-collections/amazon.aws/pull/331).
- aws_ec2 inventory - expose to new keys called ``include_filters`` and ``exclude_filters`` to give the user the ability to compose an inventory with multiple queries (https://github.com/ansible-collections/amazon.aws/pull/328).
- aws_ec2 inventory plugin - Added support for using Jinja2 templates in the authentication fields (https://github.com/ansible-collections/amazon.aws/pull/57).
- cloudformation - added support for StackPolicyDuringUpdateBody (https://github.com/ansible-collections/amazon.aws/pull/155).
- ec2_metadata_facts - add support for IMDSv2 (https://github.com/ansible-collections/amazon.aws/pull/43).
- ec2_snapshot_info - add the ``max_results`` along with ``next_token_id`` option (https://github.com/ansible-collections/amazon.aws/pull/321).
- ec2_tag - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309).
- ec2_tag_info - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309).
- ec2_vol - add the ``purge_tags`` option (https://github.com/ansible-collections/amazon.aws/pull/242).
- ec2_vol - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309).
- ec2_vpc_net - use a custom waiter which can handle API rate limiting (https://github.com/ansible-collections/amazon.aws/pull/270).
- ec2_vpc_subnet - use AWSRetry decorator to more consistently handle API rate limiting (https://github.com/ansible-collections/amazon.aws/pull/270).
- ec2_vpc_subnet - use common code for tagging resources (https://github.com/ansible-collections/amazon.aws/pull/309).
- module_utils.cloudfront_facts - linting cleanup (https://github.com/ansible-collections/amazon.aws/pull/291).
- module_utils.ec2 - linting cleanup (https://github.com/ansible-collections/amazon.aws/pull/291).
- module_utils/core - add a helper function ``normalize_boto3_result`` (https://github.com/ansible-collections/amazon.aws/pull/271).
- module_utils/core - add parameter ``descend_into_lists`` to ``scrub_none_parameters`` helper function (https://github.com/ansible-collections/amazon.aws/pull/262).
- module_utils/ec2 - added additional helper functions for tagging EC2 resources (https://github.com/ansible-collections/amazon.aws/pull/309).
- sanity tests - add ignore.txt for 2.12 (https://github.com/ansible-collections/amazon.aws/pull/315).
Bugfixes
--------
- ec2_vol - create or update now preserves the existing tags, including Name (https://github.com/ansible-collections/amazon.aws/issues/229)
- ec2_vol - fix exception when platform information isn't available (https://github.com/ansible-collections/amazon.aws/issues/305).
v1.4.1
======
Minor Changes
-------------
- module_utils - the ipaddress module utility has been vendored into this collection. This eliminates the collection dependency on ansible.netcommon (which had removed the library in its 2.0 release). The ipaddress library is provided for internal use in this collection only. (https://github.com/ansible-collections/amazon.aws/issues/273)-
v1.4.0
======
Minor Changes
-------------
- aws_ec2 - Add hostname options concatenation
- aws_ec2 inventory plugin - avoid a superfluous import of ``ansible.utils.display.Display`` (https://github.com/ansible-collections/amazon.aws/pull/226).
- aws_ec2 module - Replace inverse aws instance-state-name filters !terminated, !shutting-down in favor of postive filters pending, running, stopping, stopped. Issue 235. (https://github.com/ansible-collections/amazon.aws/pull/237)
- aws_secret - add ``bypath`` functionality (https://github.com/ansible-collections/amazon.aws/pull/192).
- ec2_key - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/213).
- ec2_vol - Add support for gp3 volumes and support for modifying existing volumes (https://github.com/ansible-collections/amazon.aws/issues/55).
- module_utils/elbv2 - add logic to compare_rules to suit Values list nested within dicts unique to each field type. Fixes issue (https://github.com/ansible-collections/amazon.aws/issues/187)
- various AWS plugins and module_utils - Cleanup unused imports (https://github.com/ansible-collections/amazon.aws/pull/217).
Bugfixes
--------
- ec2_vol - a creation or update now returns a structure with an up to date list of tags (https://github.com/ansible-collections/amazon.aws/pull/241).
v1.3.0
======
Minor Changes
-------------
- aws_caller_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/208)
- aws_s3 - Add support for uploading templated content (https://github.com/ansible-collections/amazon.aws/pull/20).
- aws_secret - add "on_missing" and "on_denied" option (https://github.com/ansible-collections/amazon.aws/pull/122).
- ec2_ami - Add retries for ratelimiting related errors (https://github.com/ansible-collections/amazon.aws/pull/195).
- ec2_ami - fixed and streamlined ``max_attempts`` logic when waiting for AMI creation to finish (https://github.com/ansible-collections/amazon.aws/pull/194).
- ec2_ami - increased default ``wait_timeout`` to 1200 seconds (https://github.com/ansible-collections/amazon.aws/pull/194).
- ec2_ami_info - Add retries for ratelimiting related errors (https://github.com/ansible-collections/amazon.aws/pull/195).
- ec2_eni - Improve reliability of the module by adding waiters and performing lookups by ENI ID rather than repeated searches (https://github.com/ansible-collections/amazon.aws/pull/180).
- ec2_eni_info - Improve reliability of the module by adding waiters and performing lookups by ENI ID rather than repeated searches (https://github.com/ansible-collections/amazon.aws/pull/180).
- ec2_group - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/207)
- ec2_group_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/207)
- ec2_snapshot_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/208)
- ec2_vol - Add automatic retries on AWS rate limit errors (https://github.com/ansible-collections/amazon.aws/pull/199).
- ec2_vol - ported ec2_vol to use boto3 (https://github.com/ansible-collections/amazon.aws/pull/53).
- ec2_vpc_dhcp_option_info - add AWSRetry decorator to automatically retry on common temporary failures (https://github.com/ansible-collections/amazon.aws/pull/208)
- module_utils/core - add helper function ``scrub_none_parameters`` to remove params set to ``None`` (https://github.com/ansible-collections/community.aws/issues/251).
- module_utils/waiters - Add retries to our waiters for the same failure codes that we retry with AWSRetry (https://github.com/ansible-collections/amazon.aws/pull/185)
- s3_bucket - Add support for managing the ``public_access`` settings (https://github.com/ansible-collections/amazon.aws/pull/171).
Bugfixes
--------
- ec2 - Code fix so module can create ec2 instances with ``ec2_volume_iops`` option (https://github.com/ansible-collections/amazon.aws/pull/177).
- ec2 - ignore terminated instances and instances that are shutting down when starting and stopping (https://github.com/ansible-collections/amazon.aws/issues/146).
- ec2_group - Fixes error handling during tagging failures (https://github.com/ansible-collections/amazon.aws/issues/210).
- ec2_group_info - Code fix so module works with Python 3.8 (make dict immutable in loop) (https://github.com/ansible-collections/amazon.aws/pull/181)
v1.2.1
======
Minor Changes
-------------
- ec2_eni - Add support for tagging.
- ec2_eni - Port ec2_eni module to boto3 and add an integration test suite.
- ec2_eni_info - Add retries on transient AWS failures.
- ec2_eni_info - Add support for providing an ENI ID.
v1.2.0
======
Minor Changes
-------------
- ec2 module_utils - Update ``ec2_connect`` (boto2) behaviour so that ``ec2_url`` overrides ``region``.
- module_utils.core - Support passing arbitrary extra keys to fail_json_aws, matching capabilities of fail_json.
Deprecated Features
-------------------
- All AWS Modules - ``aws_access_key``, ``aws_secret_key`` and ``security_token`` will be made mutually exclusive with ``profile`` after 2022-06-01.
Bugfixes
--------
- ec2 module_utils - Ensure boto3 verify parameter isn't overridden by setting a profile (https://github.com/ansible-collections/amazon.aws/issues/129)
- s3_bucket - Ceph compatibility: treat error code NoSuchTagSetError used by Ceph synonymously to NoSuchTagSet used by AWS
v1.1.0
======
Major Changes
-------------
- ec2 module_utils - The ``AWSRetry`` decorator no longer catches ``NotFound`` exceptions by default. ``NotFound`` exceptions need to be explicitly added using ``catch_extra_error_codes``. Some AWS modules may see an increase in transient failures due to AWS''s eventual consistency model.
Minor Changes
-------------
- Add ``aws_security_token``, ``aws_endpoint_url`` and ``endpoint_url`` aliases to improve AWS module parameter naming consistency.
- Add support for ``aws_ca_bundle`` to boto3 based AWS modules
- Add support for configuring boto3 profiles using ``AWS_PROFILE`` and ``AWS_DEFAULT_PROFILE``
- Added check_mode support to aws_az_info
- Added check_mode support to ec2_eni_info
- Added check_mode support to ec2_snapshot_info
- ansible_dict_to_boto3_filter_list - convert integers and bools to strings before using them in filters.
- aws_direct_connect_virtual_interface - add direct_connect_gateway_id parameter. This field is only applicable in private VIF cases (public=False) and is mutually exclusive to virtual_gateway_id.
- cloudformation - Return change_set_id in the cloudformation output if a change set was created.
- ec2 - deprecate allowing both group and group_id - currently we ignore group_id if both are passed.
- ec2_ami_info - allow integer and bool values for filtering images (https://github.com/ansible/ansible/issues/43570).
- ec2_asg - Add support for Max Instance Lifetime
- ec2_asg - Add the ability to use mixed_instance_policy in launch template driven autoscaling groups
- ec2_asg - Migrated to AnsibleAWSModule
- ec2_placement_group - make ``name`` a required field.
- ec2_vol_info - Code cleanup and use of the AWSRetry decorator to improve stability
- ec2_vpc_net - Enable IPv6 CIDR assignment
Breaking Changes / Porting Guide
--------------------------------
- aws_s3 - can now delete versioned buckets even when they are not empty - set mode to delete to delete a versioned bucket and everything in it.
Deprecated Features
-------------------
- cloudformation - The ``template_format`` option had no effect since Ansible 2.3 and will be removed after 2022-06-01
- cloudformation - the ``template_format`` option has been deprecated and will be removed in a later release. It has been ignored by the module since Ansible 2.3.
- data_pipeline - The ``version`` option had no effect and will be removed in after 2022-06-01
- ec2 - in a later release, the ``group`` and ``group_id`` options will become mutually exclusive. Currently ``group_id`` is ignored if you pass both.
- ec2_ami - The ``no_device`` alias ``NoDevice`` has been deprecated and will be removed after 2022-06-01
- ec2_ami - The ``virtual_name`` alias ``VirtualName`` has been deprecated and will be removed after 2022-06-01
- ec2_eip - The ``wait_timeout`` option had no effect and will be removed after 2022-06-01
- ec2_key - The ``wait_timeout`` option had no effect and will be removed after 2022-06-01
- ec2_key - The ``wait`` option had no effect and will be removed after 2022-06-01
- ec2_key - the ``wait_timeout`` option has been deprecated and will be removed in a later release. It has had no effect since Ansible 2.5.
- ec2_key - the ``wait`` option has been deprecated and will be removed in a later release. It has had no effect since Ansible 2.5.
- ec2_lc - The ``associate_public_ip_address`` option had no effect and will be removed after 2022-06-01
- ec2_tag - deprecate the ``list`` option in favor of ec2_tag_info
- ec2_tag - support for ``list`` as a state has been deprecated and will be removed in a later release. The ``ec2_tag_info`` can be used to fetch the tags on an EC2 resource.
Bugfixes
--------
- aws_ec2 - fix idempotency when managing tags
- aws_ec2 - fix idempotency when metrics are enable
- aws_s3 - Delete objects and delete markers so versioned buckets can be removed.
- aws_s3 - Try to wait for the bucket to exist before setting the access control list.
- cloudformation_info - Fix a KeyError returning information about the stack(s).
- ec2_asg - Ensure "wait" is honored during replace operations
- ec2_launch_template - Update output to include latest_version and default_version, matching the documentation
- ec2_transit_gateway - Use AWSRetry before ClientError is handled when describing transit gateways
- ec2_transit_gateway - fixed issue where auto_attach set to yes was not being honored (https://github.com/ansible/ansible/issues/61907)
- ec2_vol - fix filtering bug
- s3_bucket - Accept XNotImplemented response to support NetApp StorageGRID.

View File

@@ -0,0 +1,81 @@
# Contributing
## Getting Started
General information about setting up your Python environment, testing modules,
Ansible coding styles, and more can be found in the [Ansible Community Guide](
https://docs.ansible.com/ansible/latest/community/index.html).
Information about AWS SDK library usage, module utils, testing, and more can be
found in the [AWS Guidelines](https://docs.ansible.com/ansible/devel/dev_guide/platforms/aws_guidelines.html)
documentation.
## AWS Collections
There are two related collections containing AWS content (modules and plugins).
### amazon.aws
This collection contains the `module_utils` (shared libraries) used by both collections.
Content in this collection is included downstream in Red Hat Ansible Automation Platform.
Code standards, test coverage, and other supportability criteria may be higher in this collection.
The `amazon.aws` collection is an [Ansible-maintained collection](https://docs.ansible.com/ansible/devel/community/contributing_maintained_collections.html).
### community.aws
This collection contains modules and plugins contributed and maintained by the Ansible AWS
community. The `community.aws` collection is tested and generally assured to work in
conjunction with `amazon.aws`.
New modules and plugins developed by the community should be proposed to `community.aws`.
Content in this collection that is stable and meets other acceptance criteria has the potential
to be promoted and migrated into `amazon.aws`.
## Submitting Issues
All software has bugs, and the `amazon.aws` collection is no exception. When you find a bug,
you can help tremendously by [telling us about it](https://github.com/ansible-collections/amazon.aws/issues/new/choose).
If you should discover that the bug you're trying to file already exists in an issue,
you can help by verifying the behavior of the reported bug with a comment in that
issue, or by reporting any additional information
## Pull Requests
All modules MUST have integration tests for new features.
Bug fixes for modules that currently have integration tests SHOULD have tests added.
New modules should be submitted to the [community.aws](https://github.com/ansible-collections/community.aws) collection
and MUST have integration tests.
Expected test criteria:
* Resource creation under check mode
* Resource creation
* Resource creation again (idempotency) under check mode
* Resource creation again (idempotency)
* Resource modification under check mode
* Resource modification
* Resource modification again (idempotency) under check mode
* Resource modification again (idempotency)
* Resource deletion under check mode
* Resource deletion
* Resource deletion (of a non-existent resource) under check mode
* Resource deletion (of a non-existent resource)
Where modules have multiple parameters we recommend running through the 4-step modification cycle for each parameter the module accepts, as well as a modification cycle where as most, if not all, parameters are modified at the same time.
For general information on running the integration tests see the
[Integration Tests page of the Module Development Guide](https://docs.ansible.com/ansible/devel/dev_guide/testing_integration.html#testing-integration),
especially the section on configuration for cloud tests. For questions about writing tests the Ansible AWS community can
be found on Libera.Chat IRC as detailed below.
### Code of Conduct
The `amazon.aws` collection follows the Ansible project's
[Code of Conduct](https://docs.ansible.com/ansible/devel/community/code_of_conduct.html).
Please read and familiarize yourself with this document.
### IRC
Our IRC channels may require you to register your nickname. If you receive an error when you connect, see
[Libera.Chat's Nickname Registration guide](https://libera.chat/guides/registration) for instructions.
The `#ansible-aws` channel on [irc.libera.chat](https://libera.chat/) is the main and official place to discuss use and development
of the `amazon.aws` collection.

View File

@@ -0,0 +1,675 @@
GNU GENERAL PUBLIC LICENSE
Version 3, 29 June 2007
Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
Everyone is permitted to copy and distribute verbatim copies
of this license document, but changing it is not allowed.
Preamble
The GNU General Public License is a free, copyleft license for
software and other kinds of works.
The licenses for most software and other practical works are designed
to take away your freedom to share and change the works. By contrast,
the GNU General Public License is intended to guarantee your freedom to
share and change all versions of a program--to make sure it remains free
software for all its users. We, the Free Software Foundation, use the
GNU General Public License for most of our software; it applies also to
any other work released this way by its authors. You can apply it to
your programs, too.
When we speak of free software, we are referring to freedom, not
price. Our General Public Licenses are designed to make sure that you
have the freedom to distribute copies of free software (and charge for
them if you wish), that you receive source code or can get it if you
want it, that you can change the software or use pieces of it in new
free programs, and that you know you can do these things.
To protect your rights, we need to prevent others from denying you
these rights or asking you to surrender the rights. Therefore, you have
certain responsibilities if you distribute copies of the software, or if
you modify it: responsibilities to respect the freedom of others.
For example, if you distribute copies of such a program, whether
gratis or for a fee, you must pass on to the recipients the same
freedoms that you received. You must make sure that they, too, receive
or can get the source code. And you must show them these terms so they
know their rights.
Developers that use the GNU GPL protect your rights with two steps:
(1) assert copyright on the software, and (2) offer you this License
giving you legal permission to copy, distribute and/or modify it.
For the developers' and authors' protection, the GPL clearly explains
that there is no warranty for this free software. For both users' and
authors' sake, the GPL requires that modified versions be marked as
changed, so that their problems will not be attributed erroneously to
authors of previous versions.
Some devices are designed to deny users access to install or run
modified versions of the software inside them, although the manufacturer
can do so. This is fundamentally incompatible with the aim of
protecting users' freedom to change the software. The systematic
pattern of such abuse occurs in the area of products for individuals to
use, which is precisely where it is most unacceptable. Therefore, we
have designed this version of the GPL to prohibit the practice for those
products. If such problems arise substantially in other domains, we
stand ready to extend this provision to those domains in future versions
of the GPL, as needed to protect the freedom of users.
Finally, every program is threatened constantly by software patents.
States should not allow patents to restrict development and use of
software on general-purpose computers, but in those that do, we wish to
avoid the special danger that patents applied to a free program could
make it effectively proprietary. To prevent this, the GPL assures that
patents cannot be used to render the program non-free.
The precise terms and conditions for copying, distribution and
modification follow.
TERMS AND CONDITIONS
0. Definitions.
"This License" refers to version 3 of the GNU General Public License.
"Copyright" also means copyright-like laws that apply to other kinds of
works, such as semiconductor masks.
"The Program" refers to any copyrightable work licensed under this
License. Each licensee is addressed as "you". "Licensees" and
"recipients" may be individuals or organizations.
To "modify" a work means to copy from or adapt all or part of the work
in a fashion requiring copyright permission, other than the making of an
exact copy. The resulting work is called a "modified version" of the
earlier work or a work "based on" the earlier work.
A "covered work" means either the unmodified Program or a work based
on the Program.
To "propagate" a work means to do anything with it that, without
permission, would make you directly or secondarily liable for
infringement under applicable copyright law, except executing it on a
computer or modifying a private copy. Propagation includes copying,
distribution (with or without modification), making available to the
public, and in some countries other activities as well.
To "convey" a work means any kind of propagation that enables other
parties to make or receive copies. Mere interaction with a user through
a computer network, with no transfer of a copy, is not conveying.
An interactive user interface displays "Appropriate Legal Notices"
to the extent that it includes a convenient and prominently visible
feature that (1) displays an appropriate copyright notice, and (2)
tells the user that there is no warranty for the work (except to the
extent that warranties are provided), that licensees may convey the
work under this License, and how to view a copy of this License. If
the interface presents a list of user commands or options, such as a
menu, a prominent item in the list meets this criterion.
1. Source Code.
The "source code" for a work means the preferred form of the work
for making modifications to it. "Object code" means any non-source
form of a work.
A "Standard Interface" means an interface that either is an official
standard defined by a recognized standards body, or, in the case of
interfaces specified for a particular programming language, one that
is widely used among developers working in that language.
The "System Libraries" of an executable work include anything, other
than the work as a whole, that (a) is included in the normal form of
packaging a Major Component, but which is not part of that Major
Component, and (b) serves only to enable use of the work with that
Major Component, or to implement a Standard Interface for which an
implementation is available to the public in source code form. A
"Major Component", in this context, means a major essential component
(kernel, window system, and so on) of the specific operating system
(if any) on which the executable work runs, or a compiler used to
produce the work, or an object code interpreter used to run it.
The "Corresponding Source" for a work in object code form means all
the source code needed to generate, install, and (for an executable
work) run the object code and to modify the work, including scripts to
control those activities. However, it does not include the work's
System Libraries, or general-purpose tools or generally available free
programs which are used unmodified in performing those activities but
which are not part of the work. For example, Corresponding Source
includes interface definition files associated with source files for
the work, and the source code for shared libraries and dynamically
linked subprograms that the work is specifically designed to require,
such as by intimate data communication or control flow between those
subprograms and other parts of the work.
The Corresponding Source need not include anything that users
can regenerate automatically from other parts of the Corresponding
Source.
The Corresponding Source for a work in source code form is that
same work.
2. Basic Permissions.
All rights granted under this License are granted for the term of
copyright on the Program, and are irrevocable provided the stated
conditions are met. This License explicitly affirms your unlimited
permission to run the unmodified Program. The output from running a
covered work is covered by this License only if the output, given its
content, constitutes a covered work. This License acknowledges your
rights of fair use or other equivalent, as provided by copyright law.
You may make, run and propagate covered works that you do not
convey, without conditions so long as your license otherwise remains
in force. You may convey covered works to others for the sole purpose
of having them make modifications exclusively for you, or provide you
with facilities for running those works, provided that you comply with
the terms of this License in conveying all material for which you do
not control copyright. Those thus making or running the covered works
for you must do so exclusively on your behalf, under your direction
and control, on terms that prohibit them from making any copies of
your copyrighted material outside their relationship with you.
Conveying under any other circumstances is permitted solely under
the conditions stated below. Sublicensing is not allowed; section 10
makes it unnecessary.
3. Protecting Users' Legal Rights From Anti-Circumvention Law.
No covered work shall be deemed part of an effective technological
measure under any applicable law fulfilling obligations under article
11 of the WIPO copyright treaty adopted on 20 December 1996, or
similar laws prohibiting or restricting circumvention of such
measures.
When you convey a covered work, you waive any legal power to forbid
circumvention of technological measures to the extent such circumvention
is effected by exercising rights under this License with respect to
the covered work, and you disclaim any intention to limit operation or
modification of the work as a means of enforcing, against the work's
users, your or third parties' legal rights to forbid circumvention of
technological measures.
4. Conveying Verbatim Copies.
You may convey verbatim copies of the Program's source code as you
receive it, in any medium, provided that you conspicuously and
appropriately publish on each copy an appropriate copyright notice;
keep intact all notices stating that this License and any
non-permissive terms added in accord with section 7 apply to the code;
keep intact all notices of the absence of any warranty; and give all
recipients a copy of this License along with the Program.
You may charge any price or no price for each copy that you convey,
and you may offer support or warranty protection for a fee.
5. Conveying Modified Source Versions.
You may convey a work based on the Program, or the modifications to
produce it from the Program, in the form of source code under the
terms of section 4, provided that you also meet all of these conditions:
a) The work must carry prominent notices stating that you modified
it, and giving a relevant date.
b) The work must carry prominent notices stating that it is
released under this License and any conditions added under section
7. This requirement modifies the requirement in section 4 to
"keep intact all notices".
c) You must license the entire work, as a whole, under this
License to anyone who comes into possession of a copy. This
License will therefore apply, along with any applicable section 7
additional terms, to the whole of the work, and all its parts,
regardless of how they are packaged. This License gives no
permission to license the work in any other way, but it does not
invalidate such permission if you have separately received it.
d) If the work has interactive user interfaces, each must display
Appropriate Legal Notices; however, if the Program has interactive
interfaces that do not display Appropriate Legal Notices, your
work need not make them do so.
A compilation of a covered work with other separate and independent
works, which are not by their nature extensions of the covered work,
and which are not combined with it such as to form a larger program,
in or on a volume of a storage or distribution medium, is called an
"aggregate" if the compilation and its resulting copyright are not
used to limit the access or legal rights of the compilation's users
beyond what the individual works permit. Inclusion of a covered work
in an aggregate does not cause this License to apply to the other
parts of the aggregate.
6. Conveying Non-Source Forms.
You may convey a covered work in object code form under the terms
of sections 4 and 5, provided that you also convey the
machine-readable Corresponding Source under the terms of this License,
in one of these ways:
a) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by the
Corresponding Source fixed on a durable physical medium
customarily used for software interchange.
b) Convey the object code in, or embodied in, a physical product
(including a physical distribution medium), accompanied by a
written offer, valid for at least three years and valid for as
long as you offer spare parts or customer support for that product
model, to give anyone who possesses the object code either (1) a
copy of the Corresponding Source for all the software in the
product that is covered by this License, on a durable physical
medium customarily used for software interchange, for a price no
more than your reasonable cost of physically performing this
conveying of source, or (2) access to copy the
Corresponding Source from a network server at no charge.
c) Convey individual copies of the object code with a copy of the
written offer to provide the Corresponding Source. This
alternative is allowed only occasionally and noncommercially, and
only if you received the object code with such an offer, in accord
with subsection 6b.
d) Convey the object code by offering access from a designated
place (gratis or for a charge), and offer equivalent access to the
Corresponding Source in the same way through the same place at no
further charge. You need not require recipients to copy the
Corresponding Source along with the object code. If the place to
copy the object code is a network server, the Corresponding Source
may be on a different server (operated by you or a third party)
that supports equivalent copying facilities, provided you maintain
clear directions next to the object code saying where to find the
Corresponding Source. Regardless of what server hosts the
Corresponding Source, you remain obligated to ensure that it is
available for as long as needed to satisfy these requirements.
e) Convey the object code using peer-to-peer transmission, provided
you inform other peers where the object code and Corresponding
Source of the work are being offered to the general public at no
charge under subsection 6d.
A separable portion of the object code, whose source code is excluded
from the Corresponding Source as a System Library, need not be
included in conveying the object code work.
A "User Product" is either (1) a "consumer product", which means any
tangible personal property which is normally used for personal, family,
or household purposes, or (2) anything designed or sold for incorporation
into a dwelling. In determining whether a product is a consumer product,
doubtful cases shall be resolved in favor of coverage. For a particular
product received by a particular user, "normally used" refers to a
typical or common use of that class of product, regardless of the status
of the particular user or of the way in which the particular user
actually uses, or expects or is expected to use, the product. A product
is a consumer product regardless of whether the product has substantial
commercial, industrial or non-consumer uses, unless such uses represent
the only significant mode of use of the product.
"Installation Information" for a User Product means any methods,
procedures, authorization keys, or other information required to install
and execute modified versions of a covered work in that User Product from
a modified version of its Corresponding Source. The information must
suffice to ensure that the continued functioning of the modified object
code is in no case prevented or interfered with solely because
modification has been made.
If you convey an object code work under this section in, or with, or
specifically for use in, a User Product, and the conveying occurs as
part of a transaction in which the right of possession and use of the
User Product is transferred to the recipient in perpetuity or for a
fixed term (regardless of how the transaction is characterized), the
Corresponding Source conveyed under this section must be accompanied
by the Installation Information. But this requirement does not apply
if neither you nor any third party retains the ability to install
modified object code on the User Product (for example, the work has
been installed in ROM).
The requirement to provide Installation Information does not include a
requirement to continue to provide support service, warranty, or updates
for a work that has been modified or installed by the recipient, or for
the User Product in which it has been modified or installed. Access to a
network may be denied when the modification itself materially and
adversely affects the operation of the network or violates the rules and
protocols for communication across the network.
Corresponding Source conveyed, and Installation Information provided,
in accord with this section must be in a format that is publicly
documented (and with an implementation available to the public in
source code form), and must require no special password or key for
unpacking, reading or copying.
7. Additional Terms.
"Additional permissions" are terms that supplement the terms of this
License by making exceptions from one or more of its conditions.
Additional permissions that are applicable to the entire Program shall
be treated as though they were included in this License, to the extent
that they are valid under applicable law. If additional permissions
apply only to part of the Program, that part may be used separately
under those permissions, but the entire Program remains governed by
this License without regard to the additional permissions.
When you convey a copy of a covered work, you may at your option
remove any additional permissions from that copy, or from any part of
it. (Additional permissions may be written to require their own
removal in certain cases when you modify the work.) You may place
additional permissions on material, added by you to a covered work,
for which you have or can give appropriate copyright permission.
Notwithstanding any other provision of this License, for material you
add to a covered work, you may (if authorized by the copyright holders of
that material) supplement the terms of this License with terms:
a) Disclaiming warranty or limiting liability differently from the
terms of sections 15 and 16 of this License; or
b) Requiring preservation of specified reasonable legal notices or
author attributions in that material or in the Appropriate Legal
Notices displayed by works containing it; or
c) Prohibiting misrepresentation of the origin of that material, or
requiring that modified versions of such material be marked in
reasonable ways as different from the original version; or
d) Limiting the use for publicity purposes of names of licensors or
authors of the material; or
e) Declining to grant rights under trademark law for use of some
trade names, trademarks, or service marks; or
f) Requiring indemnification of licensors and authors of that
material by anyone who conveys the material (or modified versions of
it) with contractual assumptions of liability to the recipient, for
any liability that these contractual assumptions directly impose on
those licensors and authors.
All other non-permissive additional terms are considered "further
restrictions" within the meaning of section 10. If the Program as you
received it, or any part of it, contains a notice stating that it is
governed by this License along with a term that is a further
restriction, you may remove that term. If a license document contains
a further restriction but permits relicensing or conveying under this
License, you may add to a covered work material governed by the terms
of that license document, provided that the further restriction does
not survive such relicensing or conveying.
If you add terms to a covered work in accord with this section, you
must place, in the relevant source files, a statement of the
additional terms that apply to those files, or a notice indicating
where to find the applicable terms.
Additional terms, permissive or non-permissive, may be stated in the
form of a separately written license, or stated as exceptions;
the above requirements apply either way.
8. Termination.
You may not propagate or modify a covered work except as expressly
provided under this License. Any attempt otherwise to propagate or
modify it is void, and will automatically terminate your rights under
this License (including any patent licenses granted under the third
paragraph of section 11).
However, if you cease all violation of this License, then your
license from a particular copyright holder is reinstated (a)
provisionally, unless and until the copyright holder explicitly and
finally terminates your license, and (b) permanently, if the copyright
holder fails to notify you of the violation by some reasonable means
prior to 60 days after the cessation.
Moreover, your license from a particular copyright holder is
reinstated permanently if the copyright holder notifies you of the
violation by some reasonable means, this is the first time you have
received notice of violation of this License (for any work) from that
copyright holder, and you cure the violation prior to 30 days after
your receipt of the notice.
Termination of your rights under this section does not terminate the
licenses of parties who have received copies or rights from you under
this License. If your rights have been terminated and not permanently
reinstated, you do not qualify to receive new licenses for the same
material under section 10.
9. Acceptance Not Required for Having Copies.
You are not required to accept this License in order to receive or
run a copy of the Program. Ancillary propagation of a covered work
occurring solely as a consequence of using peer-to-peer transmission
to receive a copy likewise does not require acceptance. However,
nothing other than this License grants you permission to propagate or
modify any covered work. These actions infringe copyright if you do
not accept this License. Therefore, by modifying or propagating a
covered work, you indicate your acceptance of this License to do so.
10. Automatic Licensing of Downstream Recipients.
Each time you convey a covered work, the recipient automatically
receives a license from the original licensors, to run, modify and
propagate that work, subject to this License. You are not responsible
for enforcing compliance by third parties with this License.
An "entity transaction" is a transaction transferring control of an
organization, or substantially all assets of one, or subdividing an
organization, or merging organizations. If propagation of a covered
work results from an entity transaction, each party to that
transaction who receives a copy of the work also receives whatever
licenses to the work the party's predecessor in interest had or could
give under the previous paragraph, plus a right to possession of the
Corresponding Source of the work from the predecessor in interest, if
the predecessor has it or can get it with reasonable efforts.
You may not impose any further restrictions on the exercise of the
rights granted or affirmed under this License. For example, you may
not impose a license fee, royalty, or other charge for exercise of
rights granted under this License, and you may not initiate litigation
(including a cross-claim or counterclaim in a lawsuit) alleging that
any patent claim is infringed by making, using, selling, offering for
sale, or importing the Program or any portion of it.
11. Patents.
A "contributor" is a copyright holder who authorizes use under this
License of the Program or a work on which the Program is based. The
work thus licensed is called the contributor's "contributor version".
A contributor's "essential patent claims" are all patent claims
owned or controlled by the contributor, whether already acquired or
hereafter acquired, that would be infringed by some manner, permitted
by this License, of making, using, or selling its contributor version,
but do not include claims that would be infringed only as a
consequence of further modification of the contributor version. For
purposes of this definition, "control" includes the right to grant
patent sublicenses in a manner consistent with the requirements of
this License.
Each contributor grants you a non-exclusive, worldwide, royalty-free
patent license under the contributor's essential patent claims, to
make, use, sell, offer for sale, import and otherwise run, modify and
propagate the contents of its contributor version.
In the following three paragraphs, a "patent license" is any express
agreement or commitment, however denominated, not to enforce a patent
(such as an express permission to practice a patent or covenant not to
sue for patent infringement). To "grant" such a patent license to a
party means to make such an agreement or commitment not to enforce a
patent against the party.
If you convey a covered work, knowingly relying on a patent license,
and the Corresponding Source of the work is not available for anyone
to copy, free of charge and under the terms of this License, through a
publicly available network server or other readily accessible means,
then you must either (1) cause the Corresponding Source to be so
available, or (2) arrange to deprive yourself of the benefit of the
patent license for this particular work, or (3) arrange, in a manner
consistent with the requirements of this License, to extend the patent
license to downstream recipients. "Knowingly relying" means you have
actual knowledge that, but for the patent license, your conveying the
covered work in a country, or your recipient's use of the covered work
in a country, would infringe one or more identifiable patents in that
country that you have reason to believe are valid.
If, pursuant to or in connection with a single transaction or
arrangement, you convey, or propagate by procuring conveyance of, a
covered work, and grant a patent license to some of the parties
receiving the covered work authorizing them to use, propagate, modify
or convey a specific copy of the covered work, then the patent license
you grant is automatically extended to all recipients of the covered
work and works based on it.
A patent license is "discriminatory" if it does not include within
the scope of its coverage, prohibits the exercise of, or is
conditioned on the non-exercise of one or more of the rights that are
specifically granted under this License. You may not convey a covered
work if you are a party to an arrangement with a third party that is
in the business of distributing software, under which you make payment
to the third party based on the extent of your activity of conveying
the work, and under which the third party grants, to any of the
parties who would receive the covered work from you, a discriminatory
patent license (a) in connection with copies of the covered work
conveyed by you (or copies made from those copies), or (b) primarily
for and in connection with specific products or compilations that
contain the covered work, unless you entered into that arrangement,
or that patent license was granted, prior to 28 March 2007.
Nothing in this License shall be construed as excluding or limiting
any implied license or other defenses to infringement that may
otherwise be available to you under applicable patent law.
12. No Surrender of Others' Freedom.
If conditions are imposed on you (whether by court order, agreement or
otherwise) that contradict the conditions of this License, they do not
excuse you from the conditions of this License. If you cannot convey a
covered work so as to satisfy simultaneously your obligations under this
License and any other pertinent obligations, then as a consequence you may
not convey it at all. For example, if you agree to terms that obligate you
to collect a royalty for further conveying from those to whom you convey
the Program, the only way you could satisfy both those terms and this
License would be to refrain entirely from conveying the Program.
13. Use with the GNU Affero General Public License.
Notwithstanding any other provision of this License, you have
permission to link or combine any covered work with a work licensed
under version 3 of the GNU Affero General Public License into a single
combined work, and to convey the resulting work. The terms of this
License will continue to apply to the part which is the covered work,
but the special requirements of the GNU Affero General Public License,
section 13, concerning interaction through a network will apply to the
combination as such.
14. Revised Versions of this License.
The Free Software Foundation may publish revised and/or new versions of
the GNU General Public License from time to time. Such new versions will
be similar in spirit to the present version, but may differ in detail to
address new problems or concerns.
Each version is given a distinguishing version number. If the
Program specifies that a certain numbered version of the GNU General
Public License "or any later version" applies to it, you have the
option of following the terms and conditions either of that numbered
version or of any later version published by the Free Software
Foundation. If the Program does not specify a version number of the
GNU General Public License, you may choose any version ever published
by the Free Software Foundation.
If the Program specifies that a proxy can decide which future
versions of the GNU General Public License can be used, that proxy's
public statement of acceptance of a version permanently authorizes you
to choose that version for the Program.
Later license versions may give you additional or different
permissions. However, no additional obligations are imposed on any
author or copyright holder as a result of your choosing to follow a
later version.
15. Disclaimer of Warranty.
THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY
APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT
HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM "AS IS" WITHOUT WARRANTY
OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO,
THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM
IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF
ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
16. Limitation of Liability.
IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING
WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MODIFIES AND/OR CONVEYS
THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY
GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE
USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF
DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD
PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS),
EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF
SUCH DAMAGES.
17. Interpretation of Sections 15 and 16.
If the disclaimer of warranty and limitation of liability provided
above cannot be given local legal effect according to their terms,
reviewing courts shall apply local law that most closely approximates
an absolute waiver of all civil liability in connection with the
Program, unless a warranty or assumption of liability accompanies a
copy of the Program in return for a fee.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs
If you develop a new program, and you want it to be of the greatest
possible use to the public, the best way to achieve this is to make it
free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest
to attach them to the start of each source file to most effectively
state the exclusion of warranty; and each file should have at least
the "copyright" line and a pointer to where the full notice is found.
<one line to give the program's name and a brief idea of what it does.>
Copyright (C) <year> <name of author>
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>.
Also add information on how to contact you by electronic and paper mail.
If the program does terminal interaction, make it output a short
notice like this when it starts in an interactive mode:
<program> Copyright (C) <year> <name of author>
This program comes with ABSOLUTELY NO WARRANTY; for details type `show w'.
This is free software, and you are welcome to redistribute it
under certain conditions; type `show c' for details.
The hypothetical commands `show w' and `show c' should show the appropriate
parts of the General Public License. Of course, your program's commands
might be different; for a GUI interface, you would use an "about box".
You should also get your employer (if you work as a programmer) or school,
if any, to sign a "copyright disclaimer" for the program, if necessary.
For more information on this, and how to apply and follow the GNU GPL, see
<http://www.gnu.org/licenses/>.
The GNU General Public License does not permit incorporating your program
into proprietary programs. If your program is a subroutine library, you
may consider it more useful to permit linking proprietary applications with
the library. If this is what you want to do, use the GNU Lesser General
Public License instead of this License. But first, please read
<http://www.gnu.org/philosophy/why-not-lgpl.html>.

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,32 @@
{
"collection_info": {
"namespace": "amazon",
"name": "aws",
"version": "5.2.0",
"authors": [
"Ansible (https://github.com/ansible)"
],
"readme": "README.md",
"tags": [
"amazon",
"aws",
"cloud"
],
"description": null,
"license": [],
"license_file": "COPYING",
"dependencies": {},
"repository": "https://github.com/ansible-collections/amazon.aws",
"documentation": "https://ansible-collections.github.io/amazon.aws/branch/stable-5/collections/amazon/aws/index.html",
"homepage": "https://github.com/ansible-collections/amazon.aws",
"issues": "https://github.com/ansible-collections/amazon.aws/issues?q=is%3Aissue+is%3Aopen+sort%3Aupdated-desc"
},
"file_manifest_file": {
"name": "FILES.json",
"ftype": "file",
"chksum_type": "sha256",
"chksum_sha256": "9ffecb91c27369291e730080dd9a09215f53577bced41058013ae9c3af7f8bb9",
"format": 1
},
"format": 1
}

View File

@@ -0,0 +1,48 @@
PYTHON SOFTWARE FOUNDATION LICENSE VERSION 2
--------------------------------------------
1. This LICENSE AGREEMENT is between the Python Software Foundation
("PSF"), and the Individual or Organization ("Licensee") accessing and
otherwise using this software ("Python") in source or binary form and
its associated documentation.
2. Subject to the terms and conditions of this License Agreement, PSF hereby
grants Licensee a nonexclusive, royalty-free, world-wide license to reproduce,
analyze, test, perform and/or display publicly, prepare derivative works,
distribute, and otherwise use Python alone or in any derivative version,
provided, however, that PSF's License Agreement and PSF's notice of copyright,
i.e., "Copyright (c) 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, 2009, 2010,
2011, 2012, 2013, 2014, 2015, 2016, 2017, 2018, 2019, 2020, 2021 Python Software Foundation;
All Rights Reserved" are retained in Python alone or in any derivative version
prepared by Licensee.
3. In the event Licensee prepares a derivative work that is based on
or incorporates Python or any part thereof, and wants to make
the derivative work available to others as provided herein, then
Licensee hereby agrees to include in any such work a brief summary of
the changes made to Python.
4. PSF is making Python available to Licensee on an "AS IS"
basis. PSF MAKES NO REPRESENTATIONS OR WARRANTIES, EXPRESS OR
IMPLIED. BY WAY OF EXAMPLE, BUT NOT LIMITATION, PSF MAKES NO AND
DISCLAIMS ANY REPRESENTATION OR WARRANTY OF MERCHANTABILITY OR FITNESS
FOR ANY PARTICULAR PURPOSE OR THAT THE USE OF PYTHON WILL NOT
INFRINGE ANY THIRD PARTY RIGHTS.
5. PSF SHALL NOT BE LIABLE TO LICENSEE OR ANY OTHER USERS OF PYTHON
FOR ANY INCIDENTAL, SPECIAL, OR CONSEQUENTIAL DAMAGES OR LOSS AS
A RESULT OF MODIFYING, DISTRIBUTING, OR OTHERWISE USING PYTHON,
OR ANY DERIVATIVE THEREOF, EVEN IF ADVISED OF THE POSSIBILITY THEREOF.
6. This License Agreement will automatically terminate upon a material
breach of its terms and conditions.
7. Nothing in this License Agreement shall be deemed to create any
relationship of agency, partnership, or joint venture between PSF and
Licensee. This License Agreement does not grant permission to use PSF
trademarks or trade name in a trademark sense to endorse or promote
products or services of Licensee, or any third party.
8. By copying, installing or otherwise using Python, Licensee
agrees to be bound by the terms and conditions of this License
Agreement.

View File

@@ -0,0 +1,131 @@
# Amazon AWS Collection
The Ansible Amazon AWS collection includes a variety of Ansible content to help automate the management of AWS instances. This collection is maintained by the Ansible cloud team.
AWS related modules and plugins supported by the Ansible community are in the [community.aws](https://github.com/ansible-collections/community.aws/) collection.
## Ansible version compatibility
Tested with the Ansible Core 2.12, and 2.13 releases, and the current development version of Ansible. Ansible Core versions before 2.11.0 are not supported. In particular, Ansible Core 2.10 and Ansible 2.9 are not supported.
Use amazon.aws 4.x.y if you are using Ansible 2.9 or Ansible Core 2.10.
## Python version compatibility
This collection depends on the AWS SDK for Python (Boto3 and Botocore). Due to the
[AWS SDK Python Support Policy](https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/)
this collection requires Python 3.6 or greater.
Amazon have also announced the end of support for
[Python less than 3.7](https://aws.amazon.com/blogs/developer/python-support-policy-updates-for-aws-sdks-and-tools/).
As such support for Python less than 3.7 by this collection has been deprecated and will be removed in a release
after 2023-05-31.
## AWS SDK version compatibility
Starting with the 2.0.0 releases of amazon.aws and community.aws, it is generally the collection's policy to support the versions of `botocore` and `boto3` that were released 12 months prior to the most recent major collection release, following semantic versioning (for example, 2.0.0, 3.0.0).
Version 5.0.0 of this collection supports `boto3 >= 1.18.0` and `botocore >= 1.21.0`
All support for the original AWS SDK `boto` was removed in release 4.0.0.
## Included content
<!--start collection content-->
See the complete list of collection content in the [Plugin Index](https://ansible-collections.github.io/amazon.aws/branch/stable-5/collections/amazon/aws/index.html#plugin-index).
<!--end collection content-->
## Installing this collection
You can install the AWS collection with the Ansible Galaxy CLI:
ansible-galaxy collection install amazon.aws
You can also include it in a `requirements.yml` file and install it with `ansible-galaxy collection install -r requirements.yml`, using the format:
```yaml
---
collections:
- name: amazon.aws
```
A specific version of the collection can be installed by using the `version` keyword in the `requirements.yml` file:
```yaml
---
collections:
- name: amazon.aws
version: 3.1.1
```
The python module dependencies are not installed by `ansible-galaxy`. They can
be manually installed using pip:
pip install -r requirements.txt
or:
pip install boto3 botocore
## Using this collection
You can either call modules by their Fully Qualified Collection Name (FQCN), such as `amazon.aws.ec2_instance`, or you can call modules by their short name if you list the `amazon.aws` collection in the playbook's `collections` keyword:
```yaml
---
- name: Setup an instance for testing
amazon.aws.ec2_instance:
name: '{{ resource_prefix }}'
instance_type: t2.nano
image_id: "{{ (amis.images | sort(attribute='creation_date') | last).image_id }}"
wait: yes
volumes:
- device_name: /dev/xvda
ebs:
volume_size: 8
delete_on_termination: true
register: instance
```
### See Also:
* [Amazon Web Services Guide](https://docs.ansible.com/ansible/latest/scenario_guides/guide_aws.html)
* [Ansible Using collections](https://docs.ansible.com/ansible/latest/user_guide/collections_using.html) for more details.
## Contributing to this collection
We welcome community contributions to this collection. If you find problems, please open an issue or create a PR against the [Amazon AWS collection repository](https://github.com/ansible-collections/amazon.aws).
See [Contributing to Ansible-maintained collections](https://docs.ansible.com/ansible/devel/community/contributing_maintained_collections.html#contributing-maintained-collections) for more details.
You can also join us on:
- Libera.Chat IRC - the ``#ansible-aws`` [irc.libera.chat](https://libera.chat/) channel
### More information about contributing
- [Ansible Community Guide](https://docs.ansible.com/ansible/latest/community/index.html) - Details on contributing to Ansible
- [Contributing to Collections](https://docs.ansible.com/ansible/devel/dev_guide/developing_collections.html#contributing-to-collections) - How to check out collection git repositories correctly
- [Guidelines for Ansible Amazon AWS module development](https://docs.ansible.com/ansible/latest/dev_guide/platforms/aws_guidelines.html)
- [Getting Started With AWS Ansible Module Development and Community Contribution](https://www.ansible.com/blog/getting-started-with-aws-ansible-module-development)
## Release notes
See the [rendered changelog](https://ansible-collections.github.io/amazon.aws/branch/stable-5/collections/amazon/aws/docsite/CHANGELOG.html) or the [raw generated changelog](https://github.com/ansible-collections/amazon.aws/tree/stable-5/CHANGELOG.rst).
## Roadmap
<!-- Optional. Include the roadmap for this collection, and the proposed release/versioning strategy so users can anticipate the upgrade/update cycle. -->
## More information
- [Ansible Collection overview](https://github.com/ansible-collections/overview)
- [Ansible User guide](https://docs.ansible.com/ansible/latest/user_guide/index.html)
- [Ansible Developer guide](https://docs.ansible.com/ansible/latest/dev_guide/index.html)
- [Ansible Collection Developer Guide](https://docs.ansible.com/ansible/devel/dev_guide/developing_collections.html)
- [Ansible Community code of conduct](https://docs.ansible.com/ansible/latest/community/code_of_conduct.html)
## Licensing
GNU General Public License v3.0 or later.
See [COPYING](https://www.gnu.org/licenses/gpl-3.0.txt) to see the full text.

View File

@@ -0,0 +1,4 @@
# Needed by the ec2_key integration tests (generating EC2 format fingerprint)
openssl [test platform:rpm]
gcc [test platform:rpm]
python3-devel [test platform:rpm]

View File

@@ -0,0 +1,29 @@
changelog_filename_template: ../CHANGELOG.rst
changelog_filename_version_depth: 0
changes_file: changelog.yaml
changes_format: combined
keep_fragments: false
mention_ancestor: true
new_plugins_after_name: removed_features
notesdir: fragments
prelude_section_name: release_summary
prelude_section_title: Release Summary
sections:
- - major_changes
- Major Changes
- - minor_changes
- Minor Changes
- - breaking_changes
- Breaking Changes / Porting Guide
- - deprecated_features
- Deprecated Features
- - removed_features
- Removed Features (previously deprecated)
- - security_fixes
- Security Fixes
- - bugfixes
- Bugfixes
- - known_issues
- Known Issues
title: amazon.aws
trivial_section_name: trivial

View File

@@ -0,0 +1,126 @@
---
requires_ansible: '>=2.11.0'
action_groups:
aws:
- autoscaling_group
- autoscaling_group_info
- aws_az_info
- aws_caller_info
- aws_s3
- cloudformation
- cloudformation_info
- cloudtrail
- cloudtrail_info
- cloudwatch_metric_alarm
- cloudwatchevent_rule
- cloudwatchevent_rule
- cloudwatchlogs_log_group
- cloudwatchlogs_log_group_info
- cloudwatchlogs_log_group_metric_filter
- cloudwatch_metric_alarm_info
- ec2_ami
- ec2_ami_info
- ec2_eip
- ec2_eip_info
- ec2_elb_lb
- ec2_eni
- ec2_eni_info
- ec2_group
- ec2_group_info
- ec2_instance
- ec2_instance_info
- ec2_key
- ec2_security_group
- ec2_security_group_info
- ec2_snapshot
- ec2_snapshot_info
- ec2_spot_instance
- ec2_spot_instance_info
- ec2_tag
- ec2_tag_info
- ec2_vol
- ec2_vol_info
- ec2_vpc_dhcp_option
- ec2_vpc_dhcp_option_info
- ec2_vpc_endpoint
- ec2_vpc_endpoint_info
- ec2_vpc_endpoint_service_info
- ec2_vpc_igw
- ec2_vpc_igw_info
- ec2_vpc_nat_gateway
- ec2_vpc_nat_gateway_info
- ec2_vpc_net
- ec2_vpc_net_info
- ec2_vpc_route_table
- ec2_vpc_route_table_info
- ec2_vpc_subnet
- ec2_vpc_subnet_info
- elb_application_lb
- elb_application_lb_info
- elb_classic_lb
- execute_lambda
- iam_policy
- iam_policy_info
- iam_user
- iam_user_info
- kms_key
- kms_key_info
- lambda
- lambda_alias
- lambda_event
- lambda_execute
- lambda_info
- lambda_policy
- rds_cluster
- rds_cluster_info
- rds_cluster_snapshot
- rds_instance
- rds_instance_info
- rds_instance_snapshot
- rds_option_group
- rds_option_group_info
- rds_param_group
- rds_snapshot_info
- rds_subnet_group
- route53
- route53_health_check
- route53_info
- route53_zone
- s3_bucket
- s3_object
- s3_object_info
plugin_routing:
action:
aws_s3:
redirect: amazon.aws.s3_object
modules:
aws_kms:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.kms_key
aws_kms_info:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.kms_key_info
aws_s3:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.s3_object
ec2_asg:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.autoscaling_group
ec2_asg_info:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.autoscaling_group_info
ec2_elb_lb:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.elb_classic_lb
ec2_group:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.ec2_security_group
ec2_group_info:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.ec2_security_group_info
ec2_metric_alarm:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.cloudwatch_metric_alarm
execute_lambda:
# Deprecation for this alias should not *start* prior to 2024-09-01
redirect: amazon.aws.lambda_execute

View File

@@ -0,0 +1,75 @@
# (c) 2012, Michael DeHaan <michael.dehaan@gmail.com>
# (c) 2018, Will Thames <will@thames.id.au>
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import os
from ansible.errors import AnsibleError, AnsibleAction, AnsibleActionFail, AnsibleFileNotFound
from ansible.module_utils._text import to_text
from ansible.plugins.action import ActionBase
from ansible.utils.vars import merge_hash
class ActionModule(ActionBase):
TRANSFERS_FILES = True
def run(self, tmp=None, task_vars=None):
''' handler for s3_object operations
This adds the magic that means 'src' can point to both a 'remote' file
on the 'host' or in the 'files/' lookup path on the controller.
'''
self._supports_async = True
if task_vars is None:
task_vars = dict()
result = super(ActionModule, self).run(tmp, task_vars)
del tmp # tmp no longer has any effect
source = self._task.args.get('src', None)
try:
new_module_args = self._task.args.copy()
if source:
source = os.path.expanduser(source)
# For backward compatibility check if the file exists on the remote; it should take precedence
if not self._remote_file_exists(source):
try:
source = self._loader.get_real_file(self._find_needle('files', source), decrypt=False)
new_module_args['src'] = source
except AnsibleFileNotFound:
# module handles error message for nonexistent files
new_module_args['src'] = source
except AnsibleError as e:
raise AnsibleActionFail(to_text(e))
wrap_async = self._task.async_val and not self._connection.has_native_async
# execute the s3_object module with the updated args
result = merge_hash(result, self._execute_module(module_args=new_module_args, task_vars=task_vars, wrap_async=wrap_async))
if not wrap_async:
# remove a temporary path we created
self._remove_tmp_path(self._connection._shell.tmpdir)
except AnsibleAction as e:
result.update(e.result)
return result

View File

@@ -0,0 +1,71 @@
# (C) 2018 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
# Make coding more python3-ish
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: aws_resource_actions
type: aggregate
short_description: summarizes all "resource:actions" completed
description:
- Ansible callback plugin for collecting the AWS actions completed by all boto3 modules using
AnsibleAWSModule in a playbook. Botocore endpoint logs need to be enabled for those modules, which can
be done easily by setting debug_botocore_endpoint_logs to True for group/aws using module_defaults.
requirements:
- whitelisting in configuration - see examples section below for details.
'''
EXAMPLES = '''
example: >
To enable, add this to your ansible.cfg file in the defaults block
[defaults]
callback_whitelist = aws_resource_actions
sample output: >
#
# AWS ACTIONS: ['s3:PutBucketAcl', 's3:HeadObject', 's3:DeleteObject', 's3:PutObjectAcl', 's3:CreateMultipartUpload',
# 's3:DeleteBucket', 's3:GetObject', 's3:DeleteObjects', 's3:CreateBucket', 's3:CompleteMultipartUpload',
# 's3:ListObjectsV2', 's3:HeadBucket', 's3:UploadPart', 's3:PutObject']
#
sample output: >
#
# AWS ACTIONS: ['ec2:DescribeVpcAttribute', 'ec2:DescribeVpcClassicLink', 'ec2:ModifyVpcAttribute', 'ec2:CreateTags',
# 'sts:GetCallerIdentity', 'ec2:DescribeSecurityGroups', 'ec2:DescribeTags', 'ec2:DescribeVpcs', 'ec2:CreateVpc']
#
'''
from ansible.plugins.callback import CallbackBase
from ansible.module_utils._text import to_native
class CallbackModule(CallbackBase):
CALLBACK_VERSION = 2.8
CALLBACK_TYPE = 'aggregate'
CALLBACK_NAME = 'amazon.aws.aws_resource_actions'
CALLBACK_NEEDS_WHITELIST = True
def __init__(self):
self.aws_resource_actions = []
super(CallbackModule, self).__init__()
def extend_aws_resource_actions(self, result):
if result.get('resource_actions'):
self.aws_resource_actions.extend(result['resource_actions'])
def runner_on_ok(self, host, res):
self.extend_aws_resource_actions(res)
def runner_on_failed(self, host, res, ignore_errors=False):
self.extend_aws_resource_actions(res)
def v2_runner_item_on_ok(self, result):
self.extend_aws_resource_actions(result._result)
def v2_runner_item_on_failed(self, result):
self.extend_aws_resource_actions(result._result)
def playbook_on_stats(self, stats):
if self.aws_resource_actions:
self.aws_resource_actions = sorted(list(to_native(action) for action in set(self.aws_resource_actions)))
self._display.display("AWS ACTIONS: {0}".format(self.aws_resource_actions))

View File

@@ -0,0 +1,133 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2014, Will Thames <will@thames.id.au>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ModuleDocFragment(object):
# AWS only documentation fragment
DOCUMENTATION = r'''
options:
access_key:
description:
- AWS access key ID.
- See the AWS documentation for more information about access tokens
U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
- The C(AWS_ACCESS_KEY_ID), C(AWS_ACCESS_KEY) or C(EC2_ACCESS_KEY)
environment variables may also be used in decreasing order of
preference.
- The I(aws_access_key) and I(profile) options are mutually exclusive.
- The I(aws_access_key_id) alias was added in release 5.1.0 for
consistency with the AWS botocore SDK.
- The I(ec2_access_key) alias has been deprecated and will be removed in a
release after 2024-12-01.
- Support for the C(EC2_ACCESS_KEY) environment variable has been
deprecated and will be removed in a release after 2024-12-01.
type: str
aliases: ['aws_access_key_id', 'aws_access_key', 'ec2_access_key']
secret_key:
description:
- AWS secret access key.
- See the AWS documentation for more information about access tokens
U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
- The C(AWS_SECRET_ACCESS_KEY), C(AWS_SECRET_KEY), or C(EC2_SECRET_KEY)
environment variables may also be used in decreasing order of
preference.
- The I(secret_key) and I(profile) options are mutually exclusive.
- The I(aws_secret_access_key) alias was added in release 5.1.0 for
consistency with the AWS botocore SDK.
- The I(ec2_secret_key) alias has been deprecated and will be removed in a
release after 2024-12-01.
- Support for the C(EC2_SECRET_KEY) environment variable has been
deprecated and will be removed in a release after 2024-12-01.
type: str
aliases: ['aws_secret_access_key', 'aws_secret_key', 'ec2_secret_key']
session_token:
description:
- AWS STS session token for use with temporary credentials.
- See the AWS documentation for more information about access tokens
U(https://docs.aws.amazon.com/general/latest/gr/aws-sec-cred-types.html#access-keys-and-secret-access-keys).
- The C(AWS_SESSION_TOKEN), C(AWS_SECURITY_TOKEN) or C(EC2_SECURITY_TOKEN)
environment variables may also be used in decreasing order of preference.
- The I(security_token) and I(profile) options are mutually exclusive.
- Aliases I(aws_session_token) and I(session_token) were added in release
3.2.0, with the parameter being renamed from I(security_token) to
I(session_token) in release 6.0.0.
- The I(security_token), I(aws_security_token), and I(access_token)
aliases have been deprecated and will be removed in a release after
2024-12-01.
- Support for the C(EC2_SECRET_KEY) and C(AWS_SECURITY_TOKEN) environment
variables has been deprecated and will be removed in a release after
2024-12-01.
type: str
aliases: ['aws_session_token', 'security_token', 'aws_security_token', 'access_token']
profile:
description:
- A named AWS profile to use for authentication.
- See the AWS documentation for more information about named profiles
U(https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-profiles.html).
- The C(AWS_PROFILE) environment variable may also be used.
- The I(profile) option is mutually exclusive with the I(aws_access_key),
I(aws_secret_key) and I(security_token) options.
type: str
aliases: ['aws_profile']
endpoint_url:
description:
- URL to connect to instead of the default AWS endpoints. While this
can be used to connection to other AWS-compatible services the
amazon.aws and community.aws collections are only tested against
AWS.
- The C(AWS_URL) or C(EC2_URL) environment variables may also be used,
in decreasing order of preference.
- The I(ec2_url) and I(s3_url) aliases have been deprecated and will be
removed in a release after 2024-12-01.
- Support for the C(EC2_URL) environment variable has been deprecated and
will be removed in a release after 2024-12-01.
type: str
aliases: ['ec2_url', 'aws_endpoint_url', 's3_url' ]
aws_ca_bundle:
description:
- The location of a CA Bundle to use when validating SSL certificates.
- The C(AWS_CA_BUNDLE) environment variable may also be used.
type: path
validate_certs:
description:
- When set to C(false), SSL certificates will not be validated for
communication with the AWS APIs.
- Setting I(validate_certs=false) is strongly discouraged, as an
alternative, consider setting I(aws_ca_bundle) instead.
type: bool
default: true
aws_config:
description:
- A dictionary to modify the botocore configuration.
- Parameters can be found in the AWS documentation
U(https://botocore.amazonaws.com/v1/documentation/api/latest/reference/config.html#botocore.config.Config).
type: dict
debug_botocore_endpoint_logs:
description:
- Use a C(botocore.endpoint) logger to parse the unique (rather than total)
C("resource:action") API calls made during a task, outputing the set to
the resource_actions key in the task results. Use the
C(aws_resource_action) callback to output to total list made during
a playbook.
- The C(ANSIBLE_DEBUG_BOTOCORE_LOGS) environment variable may also be used.
type: bool
default: false
notes:
- B(Caution:) For modules, environment variables and configuration files are
read from the Ansible 'host' context and not the 'controller' context.
As such, files may need to be explicitly copied to the 'host'. For lookup
and connection plugins, environment variables and configuration files are
read from the Ansible 'controller' context and not the 'host' context.
- The AWS SDK (boto3) that Ansible uses may also read defaults for credentials
and other settings, such as the region, from its configuration files in the
Ansible 'host' context (typically C(~/.aws/credentials)).
See U(https://boto3.amazonaws.com/v1/documentation/api/latest/guide/credentials.html)
for more information.
'''

View File

@@ -0,0 +1,45 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2017, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ModuleDocFragment(object):
# Plugin options for AWS credentials
DOCUMENTATION = r'''
options:
aws_profile:
description: The AWS profile
type: str
aliases: [ boto_profile ]
env:
- name: AWS_DEFAULT_PROFILE
- name: AWS_PROFILE
aws_access_key:
description: The AWS access key to use.
type: str
aliases: [ aws_access_key_id ]
env:
- name: EC2_ACCESS_KEY
- name: AWS_ACCESS_KEY
- name: AWS_ACCESS_KEY_ID
aws_secret_key:
description: The AWS secret key that corresponds to the access key.
type: str
aliases: [ aws_secret_access_key ]
env:
- name: EC2_SECRET_KEY
- name: AWS_SECRET_KEY
- name: AWS_SECRET_ACCESS_KEY
aws_security_token:
description: The AWS security token if using temporary access and secret keys.
type: str
env:
- name: EC2_SECURITY_TOKEN
- name: AWS_SESSION_TOKEN
- name: AWS_SECURITY_TOKEN
'''

View File

@@ -0,0 +1,21 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2017, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ModuleDocFragment(object):
# Plugin option for AWS region
DOCUMENTATION = r'''
options:
region:
description: The region for which to create the connection.
type: str
env:
- name: EC2_REGION
- name: AWS_REGION
'''

View File

@@ -0,0 +1,19 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2022, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ModuleDocFragment(object):
# Minimum requirements for the collection
DOCUMENTATION = r'''
options: {}
requirements:
- python >= 3.6
- boto3 >= 1.18.0
- botocore >= 1.21.0
'''

View File

@@ -0,0 +1,30 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2015, Ansible, Inc
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ModuleDocFragment(object):
# EC2 only documentation fragment
DOCUMENTATION = r'''
options:
region:
description:
- The AWS region to use.
- For global services such as IAM, Route53 and CloudFront, I(region)
is ignored.
- The C(AWS_REGION) or C(EC2_REGION) environment variables may also
be used.
- See the Amazon AWS documentation for more information
U(http://docs.aws.amazon.com/general/latest/gr/rande.html#ec2_region).
- The C(ec2_region) alias has been deprecated and will be removed in
a release after 2024-12-01
- Support for the C(EC2_REGION) environment variable has been
deprecated and will be removed in a release after 2024-12-01.
type: str
aliases: [ aws_region, ec2_region ]
'''

View File

@@ -0,0 +1,62 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2022, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
class ModuleDocFragment(object):
# Standard Tagging related parameters
DOCUMENTATION = r'''
options:
tags:
description:
- A dictionary representing the tags to be applied to the resource.
- If the I(tags) parameter is not set then tags will not be modified.
type: dict
required: false
aliases: ['resource_tags']
purge_tags:
description:
- If I(purge_tags=true) and I(tags) is set, existing tags will be purged
from the resource to match exactly what is defined by I(tags) parameter.
- If the I(tags) parameter is not set then tags will not be modified, even
if I(purge_tags=True).
- Tag keys beginning with C(aws:) are reserved by Amazon and can not be
modified. As such they will be ignored for the purposes of the
I(purge_tags) parameter. See the Amazon documentation for more information
U(https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html#tag-conventions).
type: bool
default: true
required: false
'''
# Some modules had a default of purge_tags=False, this was generally
# deprecated in release 4.0.0
DEPRECATED_PURGE = r'''
options:
tags:
description:
- A dictionary representing the tags to be applied to the resource.
- If the I(tags) parameter is not set then tags will not be modified.
type: dict
required: false
aliases: ['resource_tags']
purge_tags:
description:
- If I(purge_tags=true) and I(tags) is set, existing tags will be purged
from the resource to match exactly what is defined by I(tags) parameter.
- If the I(tags) parameter is not set then tags will not be modified, even
if I(purge_tags=True).
- Tag keys beginning with C(aws:) are reserved by Amazon and can not be
modified. As such they will be ignored for the purposes of the
I(purge_tags) parameter. See the Amazon documentation for more information
U(https://docs.aws.amazon.com/general/latest/gr/aws_tagging.html#tag-conventions).
- The current default value of C(False) has been deprecated. The default
value will change to C(True) in release 5.0.0.
type: bool
required: false
'''

View File

@@ -0,0 +1,926 @@
# Copyright (c) 2017 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: aws_ec2
short_description: EC2 inventory source
extends_documentation_fragment:
- inventory_cache
- constructed
- amazon.aws.boto3
- amazon.aws.aws_credentials
description:
- Get inventory hosts from Amazon Web Services EC2.
- Uses a YAML configuration file that ends with C(aws_ec2.{yml|yaml}).
notes:
- If no credentials are provided and the control node has an associated IAM instance profile then the
role will be used for authentication.
author:
- Sloane Hertel (@s-hertel)
options:
plugin:
description: Token that ensures this is a source file for the plugin.
required: True
choices: ['aws_ec2', 'amazon.aws.aws_ec2']
iam_role_arn:
description:
- The ARN of the IAM role to assume to perform the inventory lookup. You should still provide AWS
credentials with enough privilege to perform the AssumeRole action.
regions:
description:
- A list of regions in which to describe EC2 instances.
- If empty (the default) default this will include all regions, except possibly restricted ones like us-gov-west-1 and cn-north-1.
type: list
elements: str
default: []
hostnames:
description:
- A list in order of precedence for hostname variables.
type: list
elements: dict
default: []
suboptions:
name:
description:
- Name of the host.
- Can be one of the options specified in U(http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
- To use tags as hostnames use the syntax tag:Name=Value to use the hostname Name_Value, or tag:Name to use the value of the Name tag.
- If value provided does not exist in the above options, it will be used as a literal string.
type: str
required: True
prefix:
description:
- Prefix to prepend to I(name). Same options as I(name).
- If I(prefix) is specified, final hostname will be I(prefix) + I(separator) + I(name).
type: str
default: ''
required: False
separator:
description:
- Value to separate I(prefix) and I(name) when I(prefix) is specified.
type: str
default: '_'
required: False
allow_duplicated_hosts:
description:
- By default, the first name that matches an entry of the I(hostnames) list is returned.
- Turn this flag on if you don't mind having duplicated entries in the inventory
and you want to get all the hostnames that match.
type: bool
default: False
version_added: 5.0.0
filters:
description:
- A dictionary of filter value pairs.
- Available filters are listed here U(http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
type: dict
default: {}
include_filters:
description:
- A list of filters. Any instances matching at least one of the filters are included in the result.
- Available filters are listed here U(http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
- Every entry in this list triggers a search query. As such, from a performance point of view, it's better to
keep the list as short as possible.
type: list
elements: dict
default: []
version_added: 1.5.0
exclude_filters:
description:
- A list of filters. Any instances matching one of the filters are excluded from the result.
- The filters from C(exclude_filters) take priority over the C(include_filters) and C(filters) keys
- Available filters are listed here U(http://docs.aws.amazon.com/cli/latest/reference/ec2/describe-instances.html#options).
- Every entry in this list triggers a search query. As such, from a performance point of view, it's better to
keep the list as short as possible.
type: list
elements: dict
default: []
version_added: 1.5.0
include_extra_api_calls:
description:
- Add two additional API calls for every instance to include 'persistent' and 'events' host variables.
- Spot instances may be persistent and instances may have associated events.
- The I(include_extra_api_calls) option had been deprecated and will be removed in release 6.0.0.
type: bool
default: False
strict_permissions:
description:
- By default if a 403 (Forbidden) error code is encountered this plugin will fail.
- You can set this option to False in the inventory config file which will allow 403 errors to be gracefully skipped.
type: bool
default: True
use_contrib_script_compatible_sanitization:
description:
- By default this plugin is using a general group name sanitization to create safe and usable group names for use in Ansible.
This option allows you to override that, in efforts to allow migration from the old inventory script and
matches the sanitization of groups when the script's ``replace_dash_in_groups`` option is set to ``False``.
To replicate behavior of ``replace_dash_in_groups = True`` with constructed groups,
you will need to replace hyphens with underscores via the regex_replace filter for those entries.
- For this to work you should also turn off the TRANSFORM_INVALID_GROUP_CHARS setting,
otherwise the core engine will just use the standard sanitization on top.
- This is not the default as such names break certain functionality as not all characters are valid Python identifiers
which group names end up being used as.
type: bool
default: False
use_contrib_script_compatible_ec2_tag_keys:
description:
- Expose the host tags with ec2_tag_TAGNAME keys like the old ec2.py inventory script.
- The use of this feature is discouraged and we advise to migrate to the new ``tags`` structure.
type: bool
default: False
version_added: 1.5.0
hostvars_prefix:
description:
- The prefix for host variables names coming from AWS.
type: str
version_added: 3.1.0
hostvars_suffix:
description:
- The suffix for host variables names coming from AWS.
type: str
version_added: 3.1.0
'''
EXAMPLES = '''
# Minimal example using environment vars or instance role credentials
# Fetch all hosts in us-east-1, the hostname is the public DNS if it exists, otherwise the private IP address
plugin: aws_ec2
regions:
- us-east-1
# Example using filters, ignoring permission errors, and specifying the hostname precedence
plugin: aws_ec2
# The values for profile, access key, secret key and token can be hardcoded like:
boto_profile: aws_profile
# or you could use Jinja as:
# boto_profile: "{{ lookup('env', 'AWS_PROFILE') | default('aws_profile', true) }}"
# Populate inventory with instances in these regions
regions:
- us-east-1
- us-east-2
filters:
# All instances with their `Environment` tag set to `dev`
tag:Environment: dev
# All dev and QA hosts
tag:Environment:
- dev
- qa
instance.group-id: sg-xxxxxxxx
# Ignores 403 errors rather than failing
strict_permissions: False
# Note: I(hostnames) sets the inventory_hostname. To modify ansible_host without modifying
# inventory_hostname use compose (see example below).
hostnames:
- tag:Name=Tag1,Name=Tag2 # Return specific hosts only
- tag:CustomDNSName
- dns-name
- name: 'tag:Name=Tag1,Name=Tag2'
- name: 'private-ip-address'
separator: '_'
prefix: 'tag:Name'
- name: 'test_literal' # Using literal values for hostname
separator: '-' # Hostname will be aws-test_literal
prefix: 'aws'
# Returns all the hostnames for a given instance
allow_duplicated_hosts: False
# Example using constructed features to create groups and set ansible_host
plugin: aws_ec2
regions:
- us-east-1
- us-west-1
# keyed_groups may be used to create custom groups
strict: False
keyed_groups:
# Add e.g. x86_64 hosts to an arch_x86_64 group
- prefix: arch
key: 'architecture'
# Add hosts to tag_Name_Value groups for each Name/Value tag pair
- prefix: tag
key: tags
# Add hosts to e.g. instance_type_z3_tiny
- prefix: instance_type
key: instance_type
# Create security_groups_sg_abcd1234 group for each SG
- key: 'security_groups|json_query("[].group_id")'
prefix: 'security_groups'
# Create a group for each value of the Application tag
- key: tags.Application
separator: ''
# Create a group per region e.g. aws_region_us_east_2
- key: placement.region
prefix: aws_region
# Create a group (or groups) based on the value of a custom tag "Role" and add them to a metagroup called "project"
- key: tags['Role']
prefix: foo
parent_group: "project"
# Set individual variables with compose
compose:
# Use the private IP address to connect to the host
# (note: this does not modify inventory_hostname, which is set via I(hostnames))
ansible_host: private_ip_address
# Example using include_filters and exclude_filters to compose the inventory.
plugin: aws_ec2
regions:
- us-east-1
- us-west-1
include_filters:
- tag:Name:
- 'my_second_tag'
- tag:Name:
- 'my_third_tag'
exclude_filters:
- tag:Name:
- 'my_first_tag'
# Example using groups to assign the running hosts to a group based on vpc_id
plugin: aws_ec2
boto_profile: aws_profile
# Populate inventory with instances in these regions
regions:
- us-east-2
filters:
# All instances with their state as `running`
instance-state-name: running
keyed_groups:
- prefix: tag
key: tags
compose:
ansible_host: public_dns_name
groups:
libvpc: vpc_id == 'vpc-####'
# Define prefix and suffix for host variables coming from AWS.
plugin: aws_ec2
regions:
- us-east-1
hostvars_prefix: 'aws_'
hostvars_suffix: '_ec2'
'''
import re
try:
import boto3
import botocore
except ImportError:
pass # will be captured by imported HAS_BOTO3
from ansible.errors import AnsibleError
from ansible.module_utils._text import to_native
from ansible.module_utils._text import to_text
from ansible.module_utils.basic import missing_required_lib
from ansible.plugins.inventory import BaseInventoryPlugin
from ansible.plugins.inventory import Cacheable
from ansible.plugins.inventory import Constructable
from ansible.template import Templar
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import HAS_BOTO3
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import ansible_dict_to_boto3_filter_list
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import boto3_tag_list_to_ansible_dict
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import camel_dict_to_snake_dict
from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
# The mappings give an array of keys to get from the filter name to the value
# returned by boto3's EC2 describe_instances method.
instance_meta_filter_to_boto_attr = {
'group-id': ('Groups', 'GroupId'),
'group-name': ('Groups', 'GroupName'),
'network-interface.attachment.instance-owner-id': ('OwnerId',),
'owner-id': ('OwnerId',),
'requester-id': ('RequesterId',),
'reservation-id': ('ReservationId',),
}
instance_data_filter_to_boto_attr = {
'affinity': ('Placement', 'Affinity'),
'architecture': ('Architecture',),
'availability-zone': ('Placement', 'AvailabilityZone'),
'block-device-mapping.attach-time': ('BlockDeviceMappings', 'Ebs', 'AttachTime'),
'block-device-mapping.delete-on-termination': ('BlockDeviceMappings', 'Ebs', 'DeleteOnTermination'),
'block-device-mapping.device-name': ('BlockDeviceMappings', 'DeviceName'),
'block-device-mapping.status': ('BlockDeviceMappings', 'Ebs', 'Status'),
'block-device-mapping.volume-id': ('BlockDeviceMappings', 'Ebs', 'VolumeId'),
'client-token': ('ClientToken',),
'dns-name': ('PublicDnsName',),
'host-id': ('Placement', 'HostId'),
'hypervisor': ('Hypervisor',),
'iam-instance-profile.arn': ('IamInstanceProfile', 'Arn'),
'image-id': ('ImageId',),
'instance-id': ('InstanceId',),
'instance-lifecycle': ('InstanceLifecycle',),
'instance-state-code': ('State', 'Code'),
'instance-state-name': ('State', 'Name'),
'instance-type': ('InstanceType',),
'instance.group-id': ('SecurityGroups', 'GroupId'),
'instance.group-name': ('SecurityGroups', 'GroupName'),
'ip-address': ('PublicIpAddress',),
'kernel-id': ('KernelId',),
'key-name': ('KeyName',),
'launch-index': ('AmiLaunchIndex',),
'launch-time': ('LaunchTime',),
'monitoring-state': ('Monitoring', 'State'),
'network-interface.addresses.private-ip-address': ('NetworkInterfaces', 'PrivateIpAddress'),
'network-interface.addresses.primary': ('NetworkInterfaces', 'PrivateIpAddresses', 'Primary'),
'network-interface.addresses.association.public-ip': ('NetworkInterfaces', 'PrivateIpAddresses', 'Association', 'PublicIp'),
'network-interface.addresses.association.ip-owner-id': ('NetworkInterfaces', 'PrivateIpAddresses', 'Association', 'IpOwnerId'),
'network-interface.association.public-ip': ('NetworkInterfaces', 'Association', 'PublicIp'),
'network-interface.association.ip-owner-id': ('NetworkInterfaces', 'Association', 'IpOwnerId'),
'network-interface.association.allocation-id': ('ElasticGpuAssociations', 'ElasticGpuId'),
'network-interface.association.association-id': ('ElasticGpuAssociations', 'ElasticGpuAssociationId'),
'network-interface.attachment.attachment-id': ('NetworkInterfaces', 'Attachment', 'AttachmentId'),
'network-interface.attachment.instance-id': ('InstanceId',),
'network-interface.attachment.device-index': ('NetworkInterfaces', 'Attachment', 'DeviceIndex'),
'network-interface.attachment.status': ('NetworkInterfaces', 'Attachment', 'Status'),
'network-interface.attachment.attach-time': ('NetworkInterfaces', 'Attachment', 'AttachTime'),
'network-interface.attachment.delete-on-termination': ('NetworkInterfaces', 'Attachment', 'DeleteOnTermination'),
'network-interface.availability-zone': ('Placement', 'AvailabilityZone'),
'network-interface.description': ('NetworkInterfaces', 'Description'),
'network-interface.group-id': ('NetworkInterfaces', 'Groups', 'GroupId'),
'network-interface.group-name': ('NetworkInterfaces', 'Groups', 'GroupName'),
'network-interface.ipv6-addresses.ipv6-address': ('NetworkInterfaces', 'Ipv6Addresses', 'Ipv6Address'),
'network-interface.mac-address': ('NetworkInterfaces', 'MacAddress'),
'network-interface.network-interface-id': ('NetworkInterfaces', 'NetworkInterfaceId'),
'network-interface.owner-id': ('NetworkInterfaces', 'OwnerId'),
'network-interface.private-dns-name': ('NetworkInterfaces', 'PrivateDnsName'),
# 'network-interface.requester-id': (),
'network-interface.requester-managed': ('NetworkInterfaces', 'Association', 'IpOwnerId'),
'network-interface.status': ('NetworkInterfaces', 'Status'),
'network-interface.source-dest-check': ('NetworkInterfaces', 'SourceDestCheck'),
'network-interface.subnet-id': ('NetworkInterfaces', 'SubnetId'),
'network-interface.vpc-id': ('NetworkInterfaces', 'VpcId'),
'placement-group-name': ('Placement', 'GroupName'),
'platform': ('Platform',),
'private-dns-name': ('PrivateDnsName',),
'private-ip-address': ('PrivateIpAddress',),
'product-code': ('ProductCodes', 'ProductCodeId'),
'product-code.type': ('ProductCodes', 'ProductCodeType'),
'ramdisk-id': ('RamdiskId',),
'reason': ('StateTransitionReason',),
'root-device-name': ('RootDeviceName',),
'root-device-type': ('RootDeviceType',),
'source-dest-check': ('SourceDestCheck',),
'spot-instance-request-id': ('SpotInstanceRequestId',),
'state-reason-code': ('StateReason', 'Code'),
'state-reason-message': ('StateReason', 'Message'),
'subnet-id': ('SubnetId',),
'tag': ('Tags',),
'tag-key': ('Tags',),
'tag-value': ('Tags',),
'tenancy': ('Placement', 'Tenancy'),
'virtualization-type': ('VirtualizationType',),
'vpc-id': ('VpcId',),
}
class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):
NAME = 'amazon.aws.aws_ec2'
def __init__(self):
super(InventoryModule, self).__init__()
self.group_prefix = 'aws_ec2_'
# credentials
self.boto_profile = None
self.aws_secret_access_key = None
self.aws_access_key_id = None
self.aws_security_token = None
self.iam_role_arn = None
def _compile_values(self, obj, attr):
'''
:param obj: A list or dict of instance attributes
:param attr: A key
:return The value(s) found via the attr
'''
if obj is None:
return
temp_obj = []
if isinstance(obj, list) or isinstance(obj, tuple):
for each in obj:
value = self._compile_values(each, attr)
if value:
temp_obj.append(value)
else:
temp_obj = obj.get(attr)
has_indexes = any([isinstance(temp_obj, list), isinstance(temp_obj, tuple)])
if has_indexes and len(temp_obj) == 1:
return temp_obj[0]
return temp_obj
def _get_boto_attr_chain(self, filter_name, instance):
'''
:param filter_name: The filter
:param instance: instance dict returned by boto3 ec2 describe_instances()
'''
allowed_filters = sorted(list(instance_data_filter_to_boto_attr.keys()) + list(instance_meta_filter_to_boto_attr.keys()))
# If filter not in allow_filters -> use it as a literal string
if filter_name not in allowed_filters:
return filter_name
if filter_name in instance_data_filter_to_boto_attr:
boto_attr_list = instance_data_filter_to_boto_attr[filter_name]
else:
boto_attr_list = instance_meta_filter_to_boto_attr[filter_name]
instance_value = instance
for attribute in boto_attr_list:
instance_value = self._compile_values(instance_value, attribute)
return instance_value
def _get_credentials(self):
'''
:return A dictionary of boto client credentials
'''
boto_params = {}
for credential in (('aws_access_key_id', self.aws_access_key_id),
('aws_secret_access_key', self.aws_secret_access_key),
('aws_session_token', self.aws_security_token)):
if credential[1]:
boto_params[credential[0]] = credential[1]
return boto_params
def _get_connection(self, credentials, region='us-east-1'):
try:
connection = boto3.session.Session(profile_name=self.boto_profile).client('ec2', region, **credentials)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
if self.boto_profile:
try:
connection = boto3.session.Session(profile_name=self.boto_profile).client('ec2', region)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
else:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
return connection
def _boto3_assume_role(self, credentials, region=None):
"""
Assume an IAM role passed by iam_role_arn parameter
:return: a dict containing the credentials of the assumed role
"""
iam_role_arn = self.iam_role_arn
try:
sts_connection = boto3.session.Session(profile_name=self.boto_profile).client('sts', region, **credentials)
sts_session = sts_connection.assume_role(RoleArn=iam_role_arn, RoleSessionName='ansible_aws_ec2_dynamic_inventory')
return dict(
aws_access_key_id=sts_session['Credentials']['AccessKeyId'],
aws_secret_access_key=sts_session['Credentials']['SecretAccessKey'],
aws_session_token=sts_session['Credentials']['SessionToken']
)
except botocore.exceptions.ClientError as e:
raise AnsibleError("Unable to assume IAM role: %s" % to_native(e))
def _boto3_conn(self, regions):
'''
:param regions: A list of regions to create a boto3 client
Generator that yields a boto3 client and the region
'''
credentials = self._get_credentials()
iam_role_arn = self.iam_role_arn
if not regions:
try:
# as per https://boto3.amazonaws.com/v1/documentation/api/latest/guide/ec2-example-regions-avail-zones.html
client = self._get_connection(credentials)
resp = client.describe_regions()
regions = [x['RegionName'] for x in resp.get('Regions', [])]
except botocore.exceptions.NoRegionError:
# above seems to fail depending on boto3 version, ignore and lets try something else
pass
except is_boto3_error_code('UnauthorizedOperation') as e: # pylint: disable=duplicate-except
if iam_role_arn is not None:
try:
# Describe regions assuming arn role
assumed_credentials = self._boto3_assume_role(credentials)
client = self._get_connection(assumed_credentials)
resp = client.describe_regions()
regions = [x['RegionName'] for x in resp.get('Regions', [])]
except botocore.exceptions.NoRegionError:
# above seems to fail depending on boto3 version, ignore and lets try something else
pass
else:
raise AnsibleError("Unauthorized operation: %s" % to_native(e))
# fallback to local list hardcoded in boto3 if still no regions
if not regions:
session = boto3.Session()
regions = session.get_available_regions('ec2')
# I give up, now you MUST give me regions
if not regions:
raise AnsibleError('Unable to get regions list from available methods, you must specify the "regions" option to continue.')
for region in regions:
connection = self._get_connection(credentials, region)
try:
if iam_role_arn is not None:
assumed_credentials = self._boto3_assume_role(credentials, region)
else:
assumed_credentials = credentials
connection = boto3.session.Session(profile_name=self.boto_profile).client('ec2', region, **assumed_credentials)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
if self.boto_profile:
try:
connection = boto3.session.Session(profile_name=self.boto_profile).client('ec2', region)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
else:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
yield connection, region
def _get_instances_by_region(self, regions, filters, strict_permissions):
'''
:param regions: a list of regions in which to describe instances
:param filters: a list of boto3 filter dictionaries
:param strict_permissions: a boolean determining whether to fail or ignore 403 error codes
:return A list of instance dictionaries
'''
all_instances = []
for connection, _region in self._boto3_conn(regions):
try:
# By default find non-terminated/terminating instances
if not any(f['Name'] == 'instance-state-name' for f in filters):
filters.append({'Name': 'instance-state-name', 'Values': ['running', 'pending', 'stopping', 'stopped']})
paginator = connection.get_paginator('describe_instances')
reservations = paginator.paginate(Filters=filters).build_full_result().get('Reservations')
instances = []
for r in reservations:
new_instances = r['Instances']
for instance in new_instances:
instance.update(self._get_reservation_details(r))
instances.extend(new_instances)
except botocore.exceptions.ClientError as e:
if e.response['ResponseMetadata']['HTTPStatusCode'] == 403 and not strict_permissions:
instances = []
else:
raise AnsibleError("Failed to describe instances: %s" % to_native(e))
except botocore.exceptions.BotoCoreError as e:
raise AnsibleError("Failed to describe instances: %s" % to_native(e))
all_instances.extend(instances)
return all_instances
def _get_reservation_details(self, reservation):
return {
'OwnerId': reservation['OwnerId'],
'RequesterId': reservation.get('RequesterId', ''),
'ReservationId': reservation['ReservationId']
}
@classmethod
def _get_tag_hostname(cls, preference, instance):
tag_hostnames = preference.split('tag:', 1)[1]
if ',' in tag_hostnames:
tag_hostnames = tag_hostnames.split(',')
else:
tag_hostnames = [tag_hostnames]
tags = boto3_tag_list_to_ansible_dict(instance.get('Tags', []))
tag_values = []
for v in tag_hostnames:
if '=' in v:
tag_name, tag_value = v.split('=')
if tags.get(tag_name) == tag_value:
tag_values.append(to_text(tag_name) + "_" + to_text(tag_value))
else:
tag_value = tags.get(v)
if tag_value:
tag_values.append(to_text(tag_value))
return tag_values
def _sanitize_hostname(self, hostname):
if ':' in to_text(hostname):
return self._sanitize_group_name(to_text(hostname))
else:
return to_text(hostname)
def _get_preferred_hostname(self, instance, hostnames):
'''
:param instance: an instance dict returned by boto3 ec2 describe_instances()
:param hostnames: a list of hostname destination variables in order of preference
:return the preferred identifer for the host
'''
if not hostnames:
hostnames = ['dns-name', 'private-dns-name']
hostname = None
for preference in hostnames:
if isinstance(preference, dict):
if 'name' not in preference:
raise AnsibleError("A 'name' key must be defined in a hostnames dictionary.")
hostname = self._get_preferred_hostname(instance, [preference["name"]])
hostname_from_prefix = self._get_preferred_hostname(instance, [preference["prefix"]])
separator = preference.get("separator", "_")
if hostname and hostname_from_prefix and 'prefix' in preference:
hostname = hostname_from_prefix + separator + hostname
elif preference.startswith('tag:'):
tags = self._get_tag_hostname(preference, instance)
hostname = tags[0] if tags else None
else:
hostname = self._get_boto_attr_chain(preference, instance)
if hostname:
break
if hostname:
return self._sanitize_hostname(hostname)
def get_all_hostnames(self, instance, hostnames):
'''
:param instance: an instance dict returned by boto3 ec2 describe_instances()
:param hostnames: a list of hostname destination variables
:return all the candidats matching the expectation
'''
if not hostnames:
hostnames = ['dns-name', 'private-dns-name']
hostname = None
hostname_list = []
for preference in hostnames:
if isinstance(preference, dict):
if 'name' not in preference:
raise AnsibleError("A 'name' key must be defined in a hostnames dictionary.")
hostname = self.get_all_hostnames(instance, [preference["name"]])
hostname_from_prefix = self.get_all_hostnames(instance, [preference["prefix"]])
separator = preference.get("separator", "_")
if hostname and hostname_from_prefix and 'prefix' in preference:
hostname = hostname_from_prefix[0] + separator + hostname[0]
elif preference.startswith('tag:'):
hostname = self._get_tag_hostname(preference, instance)
else:
hostname = self._get_boto_attr_chain(preference, instance)
if hostname:
if isinstance(hostname, list):
for host in hostname:
hostname_list.append(self._sanitize_hostname(host))
elif isinstance(hostname, str):
hostname_list.append(self._sanitize_hostname(hostname))
return hostname_list
def _query(self, regions, include_filters, exclude_filters, strict_permissions):
'''
:param regions: a list of regions to query
:param include_filters: a list of boto3 filter dictionaries
:param exclude_filters: a list of boto3 filter dictionaries
:param strict_permissions: a boolean determining whether to fail or ignore 403 error codes
'''
instances = []
ids_to_ignore = []
for filter in exclude_filters:
for i in self._get_instances_by_region(
regions,
ansible_dict_to_boto3_filter_list(filter),
strict_permissions):
ids_to_ignore.append(i['InstanceId'])
for filter in include_filters:
for i in self._get_instances_by_region(
regions,
ansible_dict_to_boto3_filter_list(filter),
strict_permissions):
if i['InstanceId'] not in ids_to_ignore:
instances.append(i)
ids_to_ignore.append(i['InstanceId'])
instances = sorted(instances, key=lambda x: x['InstanceId'])
return {'aws_ec2': instances}
def _populate(self, groups, hostnames, allow_duplicated_hosts=False,
hostvars_prefix=None, hostvars_suffix=None,
use_contrib_script_compatible_ec2_tag_keys=False):
for group in groups:
group = self.inventory.add_group(group)
self._add_hosts(
hosts=groups[group],
group=group,
hostnames=hostnames,
allow_duplicated_hosts=allow_duplicated_hosts,
hostvars_prefix=hostvars_prefix,
hostvars_suffix=hostvars_suffix,
use_contrib_script_compatible_ec2_tag_keys=use_contrib_script_compatible_ec2_tag_keys)
self.inventory.add_child('all', group)
@classmethod
def prepare_host_vars(cls, original_host_vars, hostvars_prefix=None, hostvars_suffix=None,
use_contrib_script_compatible_ec2_tag_keys=False):
host_vars = camel_dict_to_snake_dict(original_host_vars, ignore_list=['Tags'])
host_vars['tags'] = boto3_tag_list_to_ansible_dict(original_host_vars.get('Tags', []))
# Allow easier grouping by region
host_vars['placement']['region'] = host_vars['placement']['availability_zone'][:-1]
if use_contrib_script_compatible_ec2_tag_keys:
for k, v in host_vars['tags'].items():
host_vars["ec2_tag_%s" % k] = v
if hostvars_prefix or hostvars_suffix:
for hostvar, hostval in host_vars.copy().items():
del host_vars[hostvar]
if hostvars_prefix:
hostvar = hostvars_prefix + hostvar
if hostvars_suffix:
hostvar = hostvar + hostvars_suffix
host_vars[hostvar] = hostval
return host_vars
def iter_entry(self, hosts, hostnames, allow_duplicated_hosts=False, hostvars_prefix=None,
hostvars_suffix=None, use_contrib_script_compatible_ec2_tag_keys=False):
for host in hosts:
if allow_duplicated_hosts:
hostname_list = self.get_all_hostnames(host, hostnames)
else:
hostname_list = [self._get_preferred_hostname(host, hostnames)]
if not hostname_list or hostname_list[0] is None:
continue
host_vars = self.prepare_host_vars(
host,
hostvars_prefix,
hostvars_suffix,
use_contrib_script_compatible_ec2_tag_keys)
for name in hostname_list:
yield to_text(name), host_vars
def _add_hosts(self, hosts, group, hostnames, allow_duplicated_hosts=False,
hostvars_prefix=None, hostvars_suffix=None, use_contrib_script_compatible_ec2_tag_keys=False):
'''
:param hosts: a list of hosts to be added to a group
:param group: the name of the group to which the hosts belong
:param hostnames: a list of hostname destination variables in order of preference
:param bool allow_duplicated_hosts: if true, accept same host with different names
:param str hostvars_prefix: starts the hostvars variable name with this prefix
:param str hostvars_suffix: ends the hostvars variable name with this suffix
:param bool use_contrib_script_compatible_ec2_tag_keys: transform the host name with the legacy naming system
'''
for name, host_vars in self.iter_entry(
hosts, hostnames,
allow_duplicated_hosts=allow_duplicated_hosts,
hostvars_prefix=hostvars_prefix,
hostvars_suffix=hostvars_suffix,
use_contrib_script_compatible_ec2_tag_keys=use_contrib_script_compatible_ec2_tag_keys):
self.inventory.add_host(name, group=group)
for k, v in host_vars.items():
self.inventory.set_variable(name, k, v)
# Use constructed if applicable
strict = self.get_option('strict')
# Composed variables
self._set_composite_vars(self.get_option('compose'), host_vars, name, strict=strict)
# Complex groups based on jinja2 conditionals, hosts that meet the conditional are added to group
self._add_host_to_composed_groups(self.get_option('groups'), host_vars, name, strict=strict)
# Create groups based on variable values and add the corresponding hosts to it
self._add_host_to_keyed_groups(self.get_option('keyed_groups'), host_vars, name, strict=strict)
def _set_credentials(self, loader):
'''
:param config_data: contents of the inventory config file
'''
t = Templar(loader=loader)
credentials = {}
for credential_type in ['aws_profile', 'aws_access_key', 'aws_secret_key', 'aws_security_token', 'iam_role_arn']:
if t.is_template(self.get_option(credential_type)):
credentials[credential_type] = t.template(variable=self.get_option(credential_type), disable_lookups=False)
else:
credentials[credential_type] = self.get_option(credential_type)
self.boto_profile = credentials['aws_profile']
self.aws_access_key_id = credentials['aws_access_key']
self.aws_secret_access_key = credentials['aws_secret_key']
self.aws_security_token = credentials['aws_security_token']
self.iam_role_arn = credentials['iam_role_arn']
if not self.boto_profile and not (self.aws_access_key_id and self.aws_secret_access_key):
session = botocore.session.get_session()
try:
credentials = session.get_credentials().get_frozen_credentials()
except AttributeError:
pass
else:
self.aws_access_key_id = credentials.access_key
self.aws_secret_access_key = credentials.secret_key
self.aws_security_token = credentials.token
if not self.boto_profile and not (self.aws_access_key_id and self.aws_secret_access_key):
raise AnsibleError("Insufficient boto credentials found. Please provide them in your "
"inventory configuration file or set them as environment variables.")
def verify_file(self, path):
'''
:param loader: an ansible.parsing.dataloader.DataLoader object
:param path: the path to the inventory config file
:return the contents of the config file
'''
if super(InventoryModule, self).verify_file(path):
if path.endswith(('aws_ec2.yml', 'aws_ec2.yaml')):
return True
self.display.debug("aws_ec2 inventory filename must end with 'aws_ec2.yml' or 'aws_ec2.yaml'")
return False
def build_include_filters(self):
if self.get_option('filters'):
return [self.get_option('filters')] + self.get_option('include_filters')
elif self.get_option('include_filters'):
return self.get_option('include_filters')
else: # no filter
return [{}]
def parse(self, inventory, loader, path, cache=True):
super(InventoryModule, self).parse(inventory, loader, path)
if not HAS_BOTO3:
raise AnsibleError(missing_required_lib('botocore and boto3'))
self._read_config_data(path)
if self.get_option('use_contrib_script_compatible_sanitization'):
self._sanitize_group_name = self._legacy_script_compatible_group_sanitization
self._set_credentials(loader)
# get user specifications
regions = self.get_option('regions')
include_filters = self.build_include_filters()
exclude_filters = self.get_option('exclude_filters')
hostnames = self.get_option('hostnames')
strict_permissions = self.get_option('strict_permissions')
allow_duplicated_hosts = self.get_option('allow_duplicated_hosts')
hostvars_prefix = self.get_option("hostvars_prefix")
hostvars_suffix = self.get_option("hostvars_suffix")
use_contrib_script_compatible_ec2_tag_keys = self.get_option('use_contrib_script_compatible_ec2_tag_keys')
cache_key = self.get_cache_key(path)
# false when refresh_cache or --flush-cache is used
if cache:
# get the user-specified directive
cache = self.get_option('cache')
if self.get_option('include_extra_api_calls'):
self.display.deprecate(
"The include_extra_api_calls option has been deprecated "
" and will be removed in release 6.0.0.",
date='2024-09-01', collection_name='amazon.aws')
# Generate inventory
cache_needs_update = False
if cache:
try:
results = self._cache[cache_key]
except KeyError:
# if cache expires or cache file doesn't exist
cache_needs_update = True
if not cache or cache_needs_update:
results = self._query(regions, include_filters, exclude_filters, strict_permissions)
self._populate(
results,
hostnames,
allow_duplicated_hosts=allow_duplicated_hosts,
hostvars_prefix=hostvars_prefix,
hostvars_suffix=hostvars_suffix,
use_contrib_script_compatible_ec2_tag_keys=use_contrib_script_compatible_ec2_tag_keys)
# If the cache has expired/doesn't exist or if refresh_inventory/flush cache is used
# when the user is using caching, update the cached inventory
if cache_needs_update or (not cache and self.get_option('cache')):
self._cache[cache_key] = results
@staticmethod
def _legacy_script_compatible_group_sanitization(name):
# note that while this mirrors what the script used to do, it has many issues with unicode and usability in python
regex = re.compile(r"[^A-Za-z0-9\_\-]")
return regex.sub('_', name)

View File

@@ -0,0 +1,403 @@
# Copyright (c) 2018 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: aws_rds
short_description: RDS instance inventory source
description:
- Get instances and clusters from Amazon Web Services RDS.
- Uses a YAML configuration file that ends with aws_rds.(yml|yaml).
options:
regions:
description:
- A list of regions in which to describe RDS instances and clusters. Available regions are listed here
U(https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/Concepts.RegionsAndAvailabilityZones.html).
default: []
filters:
description:
- A dictionary of filter value pairs. Available filters are listed here
U(https://docs.aws.amazon.com/cli/latest/reference/rds/describe-db-instances.html#options). If you filter by
db-cluster-id and I(include_clusters) is True it will apply to clusters as well.
default: {}
strict_permissions:
description:
- By default if an AccessDenied exception is encountered this plugin will fail. You can set strict_permissions to
False in the inventory config file which will allow the restrictions to be gracefully skipped.
type: bool
default: True
include_clusters:
description: Whether or not to query for Aurora clusters as well as instances.
type: bool
default: False
statuses:
description: A list of desired states for instances/clusters to be added to inventory. Set to ['all'] as a shorthand to find everything.
type: list
elements: str
default:
- creating
- available
iam_role_arn:
description:
- The ARN of the IAM role to assume to perform the inventory lookup. You should still provide
AWS credentials with enough privilege to perform the AssumeRole action.
hostvars_prefix:
description:
- The prefix for host variables names coming from AWS.
type: str
version_added: 3.1.0
hostvars_suffix:
description:
- The suffix for host variables names coming from AWS.
type: str
version_added: 3.1.0
notes:
- Ansible versions prior to 2.10 should use the fully qualified plugin name 'amazon.aws.aws_rds'.
extends_documentation_fragment:
- inventory_cache
- constructed
- amazon.aws.boto3
- amazon.aws.aws_credentials
author:
- Sloane Hertel (@s-hertel)
'''
EXAMPLES = '''
plugin: aws_rds
regions:
- us-east-1
- ca-central-1
keyed_groups:
- key: 'db_parameter_groups|json_query("[].db_parameter_group_name")'
prefix: rds_parameter_group
- key: engine
prefix: rds
- key: tags
- key: region
hostvars_prefix: aws_
hostvars_suffix: _rds
'''
try:
import boto3
import botocore
except ImportError:
pass # will be captured by imported HAS_BOTO3
from ansible.errors import AnsibleError
from ansible.module_utils._text import to_native
from ansible.module_utils.basic import missing_required_lib
from ansible.plugins.inventory import BaseInventoryPlugin
from ansible.plugins.inventory import Cacheable
from ansible.plugins.inventory import Constructable
from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import HAS_BOTO3
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import ansible_dict_to_boto3_filter_list
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import boto3_tag_list_to_ansible_dict
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import camel_dict_to_snake_dict
class InventoryModule(BaseInventoryPlugin, Constructable, Cacheable):
NAME = 'amazon.aws.aws_rds'
def __init__(self):
super(InventoryModule, self).__init__()
self.credentials = {}
self.boto_profile = None
self.iam_role_arn = None
def _get_connection(self, credentials, region='us-east-1'):
try:
connection = boto3.session.Session(profile_name=self.boto_profile).client('rds', region, **credentials)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
if self.boto_profile:
try:
connection = boto3.session.Session(profile_name=self.boto_profile).client('rds', region)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
else:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
return connection
def _boto3_assume_role(self, credentials, region):
"""
Assume an IAM role passed by iam_role_arn parameter
:return: a dict containing the credentials of the assumed role
"""
iam_role_arn = self.iam_role_arn
try:
sts_connection = boto3.session.Session(profile_name=self.boto_profile).client('sts', region, **credentials)
sts_session = sts_connection.assume_role(RoleArn=iam_role_arn, RoleSessionName='ansible_aws_rds_dynamic_inventory')
return dict(
aws_access_key_id=sts_session['Credentials']['AccessKeyId'],
aws_secret_access_key=sts_session['Credentials']['SecretAccessKey'],
aws_session_token=sts_session['Credentials']['SessionToken']
)
except botocore.exceptions.ClientError as e:
raise AnsibleError("Unable to assume IAM role: %s" % to_native(e))
def _boto3_conn(self, regions):
'''
:param regions: A list of regions to create a boto3 client
Generator that yields a boto3 client and the region
'''
iam_role_arn = self.iam_role_arn
credentials = self.credentials
for region in regions:
try:
if iam_role_arn is not None:
assumed_credentials = self._boto3_assume_role(credentials, region)
else:
assumed_credentials = credentials
connection = boto3.session.Session(profile_name=self.boto_profile).client('rds', region, **assumed_credentials)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
if self.boto_profile:
try:
connection = boto3.session.Session(profile_name=self.boto_profile).client('rds', region)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError) as e:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
else:
raise AnsibleError("Insufficient credentials found: %s" % to_native(e))
yield connection, region
def _get_hosts_by_region(self, connection, filters, strict):
def _add_tags_for_hosts(connection, hosts, strict):
for host in hosts:
if 'DBInstanceArn' in host:
resource_arn = host['DBInstanceArn']
else:
resource_arn = host['DBClusterArn']
try:
tags = connection.list_tags_for_resource(ResourceName=resource_arn)['TagList']
except is_boto3_error_code('AccessDenied') as e:
if not strict:
tags = []
else:
raise e
host['Tags'] = tags
def wrapper(f, *args, **kwargs):
try:
results = f(*args, **kwargs)
if 'DBInstances' in results:
results = results['DBInstances']
else:
results = results['DBClusters']
_add_tags_for_hosts(connection, results, strict)
except is_boto3_error_code('AccessDenied') as e: # pylint: disable=duplicate-except
if not strict:
results = []
else:
raise AnsibleError("Failed to query RDS: {0}".format(to_native(e)))
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e: # pylint: disable=duplicate-except
raise AnsibleError("Failed to query RDS: {0}".format(to_native(e)))
return results
return wrapper
def _get_all_hosts(self, regions, instance_filters, cluster_filters, strict, statuses, gather_clusters=False):
'''
:param regions: a list of regions in which to describe hosts
:param instance_filters: a list of boto3 filter dictionaries
:param cluster_filters: a list of boto3 filter dictionaries
:param strict: a boolean determining whether to fail or ignore 403 error codes
:param statuses: a list of statuses that the returned hosts should match
:return A list of host dictionaries
'''
all_instances = []
all_clusters = []
for connection, _region in self._boto3_conn(regions):
paginator = connection.get_paginator('describe_db_instances')
all_instances.extend(
self._get_hosts_by_region(connection, instance_filters, strict)
(paginator.paginate(Filters=instance_filters).build_full_result)
)
if gather_clusters:
all_clusters.extend(
self._get_hosts_by_region(connection, cluster_filters, strict)
(connection.describe_db_clusters, **{'Filters': cluster_filters})
)
sorted_hosts = list(
sorted(all_instances, key=lambda x: x['DBInstanceIdentifier']) +
sorted(all_clusters, key=lambda x: x['DBClusterIdentifier'])
)
return self.find_hosts_with_valid_statuses(sorted_hosts, statuses)
def find_hosts_with_valid_statuses(self, hosts, statuses):
if 'all' in statuses:
return hosts
valid_hosts = []
for host in hosts:
if host.get('DBInstanceStatus') in statuses:
valid_hosts.append(host)
elif host.get('Status') in statuses:
valid_hosts.append(host)
return valid_hosts
def _populate(self, hosts):
group = 'aws_rds'
self.inventory.add_group(group)
if hosts:
self._add_hosts(hosts=hosts, group=group)
self.inventory.add_child('all', group)
def _populate_from_source(self, source_data):
hostvars = source_data.pop('_meta', {}).get('hostvars', {})
for group in source_data:
if group == 'all':
continue
else:
self.inventory.add_group(group)
hosts = source_data[group].get('hosts', [])
for host in hosts:
self._populate_host_vars([host], hostvars.get(host, {}), group)
self.inventory.add_child('all', group)
def _get_hostname(self, host):
if host.get('DBInstanceIdentifier'):
return host['DBInstanceIdentifier']
else:
return host['DBClusterIdentifier']
def _format_inventory(self, hosts):
results = {'_meta': {'hostvars': {}}}
group = 'aws_rds'
results[group] = {'hosts': []}
for host in hosts:
hostname = self._get_hostname(host)
results[group]['hosts'].append(hostname)
h = self.inventory.get_host(hostname)
results['_meta']['hostvars'][h.name] = h.vars
return results
def _add_hosts(self, hosts, group):
'''
:param hosts: a list of hosts to be added to a group
:param group: the name of the group to which the hosts belong
'''
for host in hosts:
hostname = self._get_hostname(host)
host = camel_dict_to_snake_dict(host, ignore_list=['Tags'])
host['tags'] = boto3_tag_list_to_ansible_dict(host.get('tags', []))
# Allow easier grouping by region
if 'availability_zone' in host:
host['region'] = host['availability_zone'][:-1]
elif 'availability_zones' in host:
host['region'] = host['availability_zones'][0][:-1]
self.inventory.add_host(hostname, group=group)
hostvars_prefix = self.get_option("hostvars_prefix")
hostvars_suffix = self.get_option("hostvars_suffix")
new_vars = dict()
for hostvar, hostval in host.items():
if hostvars_prefix:
hostvar = hostvars_prefix + hostvar
if hostvars_suffix:
hostvar = hostvar + hostvars_suffix
new_vars[hostvar] = hostval
self.inventory.set_variable(hostname, hostvar, hostval)
host.update(new_vars)
# Use constructed if applicable
strict = self.get_option('strict')
# Composed variables
self._set_composite_vars(self.get_option('compose'), host, hostname, strict=strict)
# Complex groups based on jinja2 conditionals, hosts that meet the conditional are added to group
self._add_host_to_composed_groups(self.get_option('groups'), host, hostname, strict=strict)
# Create groups based on variable values and add the corresponding hosts to it
self._add_host_to_keyed_groups(self.get_option('keyed_groups'), host, hostname, strict=strict)
def _set_credentials(self):
'''
'''
self.boto_profile = self.get_option('aws_profile')
aws_access_key_id = self.get_option('aws_access_key')
aws_secret_access_key = self.get_option('aws_secret_key')
aws_security_token = self.get_option('aws_security_token')
self.iam_role_arn = self.get_option('iam_role_arn')
if not self.boto_profile and not (aws_access_key_id and aws_secret_access_key):
session = botocore.session.get_session()
if session.get_credentials() is not None:
aws_access_key_id = session.get_credentials().access_key
aws_secret_access_key = session.get_credentials().secret_key
aws_security_token = session.get_credentials().token
if not self.boto_profile and not (aws_access_key_id and aws_secret_access_key):
raise AnsibleError("Insufficient boto credentials found. Please provide them in your "
"inventory configuration file or set them as environment variables.")
if aws_access_key_id:
self.credentials['aws_access_key_id'] = aws_access_key_id
if aws_secret_access_key:
self.credentials['aws_secret_access_key'] = aws_secret_access_key
if aws_security_token:
self.credentials['aws_session_token'] = aws_security_token
def verify_file(self, path):
'''
:param loader: an ansible.parsing.dataloader.DataLoader object
:param path: the path to the inventory config file
:return the contents of the config file
'''
if super(InventoryModule, self).verify_file(path):
if path.endswith(('aws_rds.yml', 'aws_rds.yaml')):
return True
return False
def parse(self, inventory, loader, path, cache=True):
super(InventoryModule, self).parse(inventory, loader, path)
if not HAS_BOTO3:
raise AnsibleError(missing_required_lib('botocore and boto3'))
self._read_config_data(path)
self._set_credentials()
# get user specifications
regions = self.get_option('regions')
filters = self.get_option('filters')
strict_permissions = self.get_option('strict_permissions')
statuses = self.get_option('statuses')
include_clusters = self.get_option('include_clusters')
instance_filters = ansible_dict_to_boto3_filter_list(filters)
cluster_filters = []
if 'db-cluster-id' in filters and include_clusters:
cluster_filters = ansible_dict_to_boto3_filter_list({'db-cluster-id': filters['db-cluster-id']})
cache_key = self.get_cache_key(path)
# false when refresh_cache or --flush-cache is used
if cache:
# get the user-specified directive
cache = self.get_option('cache')
# Generate inventory
formatted_inventory = {}
cache_needs_update = False
if cache:
try:
results = self._cache[cache_key]
except KeyError:
# if cache expires or cache file doesn't exist
cache_needs_update = True
else:
self._populate_from_source(results)
if not cache or cache_needs_update:
results = self._get_all_hosts(regions, instance_filters, cluster_filters, strict_permissions, statuses, include_clusters)
self._populate(results)
formatted_inventory = self._format_inventory(results)
# If the cache has expired/doesn't exist or if refresh_inventory/flush cache is used
# when the user is using caching, update the cached inventory
if cache_needs_update or (not cache and self.get_option('cache')):
self._cache[cache_key] = formatted_inventory

View File

@@ -0,0 +1,136 @@
# (c) 2017 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: aws_account_attribute
author:
- Sloane Hertel (@s-hertel) <shertel@redhat.com>
extends_documentation_fragment:
- amazon.aws.boto3
- amazon.aws.aws_credentials
- amazon.aws.aws_region
short_description: Look up AWS account attributes
description:
- Describes attributes of your AWS account. You can specify one of the listed
attribute choices or omit it to see all attributes.
options:
attribute:
description: The attribute for which to get the value(s).
choices:
- supported-platforms
- default-vpc
- max-instances
- vpc-max-security-groups-per-interface
- max-elastic-ips
- vpc-max-elastic-ips
- has-ec2-classic
'''
EXAMPLES = """
vars:
has_ec2_classic: "{{ lookup('aws_account_attribute', attribute='has-ec2-classic') }}"
# true | false
default_vpc_id: "{{ lookup('aws_account_attribute', attribute='default-vpc') }}"
# vpc-xxxxxxxx | none
account_details: "{{ lookup('aws_account_attribute', wantlist='true') }}"
# {'default-vpc': ['vpc-xxxxxxxx'], 'max-elastic-ips': ['5'], 'max-instances': ['20'],
# 'supported-platforms': ['VPC', 'EC2'], 'vpc-max-elastic-ips': ['5'], 'vpc-max-security-groups-per-interface': ['5']}
"""
RETURN = """
_raw:
description:
Returns a boolean when I(attribute) is check_ec2_classic. Otherwise returns the value(s) of the attribute
(or all attributes if one is not specified).
"""
try:
import boto3
import botocore
except ImportError:
pass # will be captured by imported HAS_BOTO3
from ansible.errors import AnsibleError
from ansible.module_utils._text import to_native
from ansible.module_utils.basic import missing_required_lib
from ansible.plugins.lookup import LookupBase
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import AWSRetry
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import HAS_BOTO3
def _boto3_conn(region, credentials):
boto_profile = credentials.pop('aws_profile', None)
try:
connection = boto3.session.Session(profile_name=boto_profile).client('ec2', region, **credentials)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError):
if boto_profile:
try:
connection = boto3.session.Session(profile_name=boto_profile).client('ec2', region)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError):
raise AnsibleError("Insufficient credentials found.")
else:
raise AnsibleError("Insufficient credentials found.")
return connection
def _get_credentials(options):
credentials = {}
credentials['aws_profile'] = options['aws_profile']
credentials['aws_secret_access_key'] = options['aws_secret_key']
credentials['aws_access_key_id'] = options['aws_access_key']
if options['aws_security_token']:
credentials['aws_session_token'] = options['aws_security_token']
return credentials
@AWSRetry.jittered_backoff(retries=10)
def _describe_account_attributes(client, **params):
return client.describe_account_attributes(**params)
class LookupModule(LookupBase):
def run(self, terms, variables, **kwargs):
if not HAS_BOTO3:
raise AnsibleError(missing_required_lib('botocore and boto3'))
self.set_options(var_options=variables, direct=kwargs)
boto_credentials = _get_credentials(self._options)
region = self._options['region']
client = _boto3_conn(region, boto_credentials)
attribute = kwargs.get('attribute')
params = {'AttributeNames': []}
check_ec2_classic = False
if 'has-ec2-classic' == attribute:
check_ec2_classic = True
params['AttributeNames'] = ['supported-platforms']
elif attribute:
params['AttributeNames'] = [attribute]
try:
response = _describe_account_attributes(client, **params)['AccountAttributes']
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
raise AnsibleError("Failed to describe account attributes: %s" % to_native(e))
if check_ec2_classic:
attr = response[0]
return any(value['AttributeValue'] == 'EC2' for value in attr['AttributeValues'])
if attribute:
attr = response[0]
return [value['AttributeValue'] for value in attr['AttributeValues']]
flattened = {}
for k_v_dict in response:
flattened[k_v_dict['AttributeName']] = [value['AttributeValue'] for value in k_v_dict['AttributeValues']]
return flattened

View File

@@ -0,0 +1,295 @@
# Copyright: (c) 2018, Aaron Smith <ajsmith10381@gmail.com>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = r'''
name: aws_secret
author:
- Aaron Smith (!UNKNOWN) <ajsmith10381@gmail.com>
extends_documentation_fragment:
- amazon.aws.boto3
- amazon.aws.aws_credentials
- amazon.aws.aws_region
short_description: Look up secrets stored in AWS Secrets Manager
description:
- Look up secrets stored in AWS Secrets Manager provided the caller
has the appropriate permissions to read the secret.
- Lookup is based on the secret's I(Name) value.
- Optional parameters can be passed into this lookup; I(version_id) and I(version_stage)
options:
_terms:
description: Name of the secret to look up in AWS Secrets Manager.
required: True
bypath:
description: A boolean to indicate whether the parameter is provided as a hierarchy.
default: false
type: boolean
version_added: 1.4.0
nested:
description: A boolean to indicate the secret contains nested values.
type: boolean
default: false
version_added: 1.4.0
version_id:
description: Version of the secret(s).
required: False
version_stage:
description: Stage of the secret version.
required: False
join:
description:
- Join two or more entries to form an extended secret.
- This is useful for overcoming the 4096 character limit imposed by AWS.
- No effect when used with I(bypath).
type: boolean
default: false
on_deleted:
description:
- Action to take if the secret has been marked for deletion.
- C(error) will raise a fatal error when the secret has been marked for deletion.
- C(skip) will silently ignore the deleted secret.
- C(warn) will skip over the deleted secret but issue a warning.
default: error
type: string
choices: ['error', 'skip', 'warn']
version_added: 2.0.0
on_missing:
description:
- Action to take if the secret is missing.
- C(error) will raise a fatal error when the secret is missing.
- C(skip) will silently ignore the missing secret.
- C(warn) will skip over the missing secret but issue a warning.
default: error
type: string
choices: ['error', 'skip', 'warn']
on_denied:
description:
- Action to take if access to the secret is denied.
- C(error) will raise a fatal error when access to the secret is denied.
- C(skip) will silently ignore the denied secret.
- C(warn) will skip over the denied secret but issue a warning.
default: error
type: string
choices: ['error', 'skip', 'warn']
'''
EXAMPLES = r"""
- name: lookup secretsmanager secret in the current region
debug: msg="{{ lookup('amazon.aws.aws_secret', '/path/to/secrets', bypath=true) }}"
- name: Create RDS instance with aws_secret lookup for password param
rds:
command: create
instance_name: app-db
db_engine: MySQL
size: 10
instance_type: db.m1.small
username: dbadmin
password: "{{ lookup('amazon.aws.aws_secret', 'DbSecret') }}"
tags:
Environment: staging
- name: skip if secret does not exist
debug: msg="{{ lookup('amazon.aws.aws_secret', 'secret-not-exist', on_missing='skip')}}"
- name: warn if access to the secret is denied
debug: msg="{{ lookup('amazon.aws.aws_secret', 'secret-denied', on_denied='warn')}}"
- name: lookup secretsmanager secret in the current region using the nested feature
debug: msg="{{ lookup('amazon.aws.aws_secret', 'secrets.environments.production.password', nested=true) }}"
# The secret can be queried using the following syntax: `aws_secret_object_name.key1.key2.key3`.
# If an object is of the form `{"key1":{"key2":{"key3":1}}}` the query would return the value `1`.
- name: lookup secretsmanager secret in a specific region using specified region and aws profile using nested feature
debug: >
msg="{{ lookup('amazon.aws.aws_secret', 'secrets.environments.production.password', region=region, aws_profile=aws_profile,
aws_access_key=aws_access_key, aws_secret_key=aws_secret_key, nested=true) }}"
# The secret can be queried using the following syntax: `aws_secret_object_name.key1.key2.key3`.
# If an object is of the form `{"key1":{"key2":{"key3":1}}}` the query would return the value `1`.
# Region is the AWS region where the AWS secret is stored.
# AWS_profile is the aws profile to use, that has access to the AWS secret.
"""
RETURN = r"""
_raw:
description:
Returns the value of the secret stored in AWS Secrets Manager.
"""
import json
try:
import boto3
import botocore
except ImportError:
pass # will be captured by imported HAS_BOTO3
from ansible.errors import AnsibleError
from ansible.module_utils.six import string_types
from ansible.module_utils._text import to_native
from ansible.module_utils.basic import missing_required_lib
from ansible.plugins.lookup import LookupBase
from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_message
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import HAS_BOTO3
def _boto3_conn(region, credentials):
boto_profile = credentials.pop('aws_profile', None)
try:
connection = boto3.session.Session(profile_name=boto_profile).client('secretsmanager', region, **credentials)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError):
if boto_profile:
try:
connection = boto3.session.Session(profile_name=boto_profile).client('secretsmanager', region)
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError):
raise AnsibleError("Insufficient credentials found.")
else:
raise AnsibleError("Insufficient credentials found.")
return connection
class LookupModule(LookupBase):
def run(self, terms, variables=None, boto_profile=None, aws_profile=None,
aws_secret_key=None, aws_access_key=None, aws_security_token=None, region=None,
bypath=False, nested=False, join=False, version_stage=None, version_id=None, on_missing='error',
on_denied='error', on_deleted='error'):
'''
:arg terms: a list of lookups to run.
e.g. ['parameter_name', 'parameter_name_too' ]
:kwarg variables: ansible variables active at the time of the lookup
:kwarg aws_secret_key: identity of the AWS key to use
:kwarg aws_access_key: AWS secret key (matching identity)
:kwarg aws_security_token: AWS session key if using STS
:kwarg decrypt: Set to True to get decrypted parameters
:kwarg region: AWS region in which to do the lookup
:kwarg bypath: Set to True to do a lookup of variables under a path
:kwarg nested: Set to True to do a lookup of nested secrets
:kwarg join: Join two or more entries to form an extended secret
:kwarg version_stage: Stage of the secret version
:kwarg version_id: Version of the secret(s)
:kwarg on_missing: Action to take if the secret is missing
:kwarg on_deleted: Action to take if the secret is marked for deletion
:kwarg on_denied: Action to take if access to the secret is denied
:returns: A list of parameter values or a list of dictionaries if bypath=True.
'''
if not HAS_BOTO3:
raise AnsibleError(missing_required_lib('botocore and boto3'))
deleted = on_deleted.lower()
if not isinstance(deleted, string_types) or deleted not in ['error', 'warn', 'skip']:
raise AnsibleError('"on_deleted" must be a string and one of "error", "warn" or "skip", not %s' % deleted)
missing = on_missing.lower()
if not isinstance(missing, string_types) or missing not in ['error', 'warn', 'skip']:
raise AnsibleError('"on_missing" must be a string and one of "error", "warn" or "skip", not %s' % missing)
denied = on_denied.lower()
if not isinstance(denied, string_types) or denied not in ['error', 'warn', 'skip']:
raise AnsibleError('"on_denied" must be a string and one of "error", "warn" or "skip", not %s' % denied)
credentials = {}
if aws_profile:
credentials['aws_profile'] = aws_profile
else:
credentials['aws_profile'] = boto_profile
credentials['aws_secret_access_key'] = aws_secret_key
credentials['aws_access_key_id'] = aws_access_key
credentials['aws_session_token'] = aws_security_token
# fallback to IAM role credentials
if not credentials['aws_profile'] and not (
credentials['aws_access_key_id'] and credentials['aws_secret_access_key']):
session = botocore.session.get_session()
if session.get_credentials() is not None:
credentials['aws_access_key_id'] = session.get_credentials().access_key
credentials['aws_secret_access_key'] = session.get_credentials().secret_key
credentials['aws_session_token'] = session.get_credentials().token
client = _boto3_conn(region, credentials)
if bypath:
secrets = {}
for term in terms:
try:
paginator = client.get_paginator('list_secrets')
paginator_response = paginator.paginate(
Filters=[{'Key': 'name', 'Values': [term]}])
for object in paginator_response:
if 'SecretList' in object:
for secret_obj in object['SecretList']:
secrets.update({secret_obj['Name']: self.get_secret_value(
secret_obj['Name'], client, on_missing=missing, on_denied=denied)})
secrets = [secrets]
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
raise AnsibleError("Failed to retrieve secret: %s" % to_native(e))
else:
secrets = []
for term in terms:
value = self.get_secret_value(term, client,
version_stage=version_stage, version_id=version_id,
on_missing=missing, on_denied=denied, on_deleted=deleted,
nested=nested)
if value:
secrets.append(value)
if join:
joined_secret = []
joined_secret.append(''.join(secrets))
return joined_secret
return secrets
def get_secret_value(self, term, client, version_stage=None, version_id=None, on_missing=None, on_denied=None, on_deleted=None, nested=False):
params = {}
params['SecretId'] = term
if version_id:
params['VersionId'] = version_id
if version_stage:
params['VersionStage'] = version_stage
if nested:
if len(term.split('.')) < 2:
raise AnsibleError("Nested query must use the following syntax: `aws_secret_name.<key_name>.<key_name>")
secret_name = term.split('.')[0]
params['SecretId'] = secret_name
try:
response = client.get_secret_value(**params)
if 'SecretBinary' in response:
return response['SecretBinary']
if 'SecretString' in response:
if nested:
query = term.split('.')[1:]
secret_string = json.loads(response['SecretString'])
ret_val = secret_string
for key in query:
if key in ret_val:
ret_val = ret_val[key]
else:
raise AnsibleError("Successfully retrieved secret but there exists no key {0} in the secret".format(key))
return str(ret_val)
else:
return response['SecretString']
except is_boto3_error_message('marked for deletion'):
if on_deleted == 'error':
raise AnsibleError("Failed to find secret %s (marked for deletion)" % term)
elif on_deleted == 'warn':
self._display.warning('Skipping, did not find secret (marked for deletion) %s' % term)
except is_boto3_error_code('ResourceNotFoundException'): # pylint: disable=duplicate-except
if on_missing == 'error':
raise AnsibleError("Failed to find secret %s (ResourceNotFound)" % term)
elif on_missing == 'warn':
self._display.warning('Skipping, did not find secret %s' % term)
except is_boto3_error_code('AccessDeniedException'): # pylint: disable=duplicate-except
if on_denied == 'error':
raise AnsibleError("Failed to access secret %s (AccessDenied)" % term)
elif on_denied == 'warn':
self._display.warning('Skipping, access denied for secret %s' % term)
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e: # pylint: disable=duplicate-except
raise AnsibleError("Failed to retrieve secret: %s" % to_native(e))
return None

View File

@@ -0,0 +1,90 @@
# (c) 2016 James Turner <turnerjsm@gmail.com>
# (c) 2017 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: aws_service_ip_ranges
author:
- James Turner (!UNKNOWN) <turnerjsm@gmail.com>
requirements:
- must have public internet connectivity
short_description: Look up the IP ranges for services provided in AWS such as EC2 and S3.
description:
- AWS publishes IP ranges used on the public internet by EC2, S3, CloudFront, CodeBuild, Route53, and Route53 Health Checking.
- This module produces a list of all the ranges (by default) or can narrow down the list to the specified region or service.
options:
service:
description: 'The service to filter ranges by. Options: EC2, S3, CLOUDFRONT, CODEbUILD, ROUTE53, ROUTE53_HEALTHCHECKS'
region:
description: 'The AWS region to narrow the ranges to. Examples: us-east-1, eu-west-2, ap-southeast-1'
ipv6_prefixes:
description: 'When I(ipv6_prefixes=True) the lookup will return ipv6 addresses instead of ipv4 addresses'
version_added: 2.1.0
'''
EXAMPLES = """
vars:
ec2_ranges: "{{ lookup('aws_service_ip_ranges', region='ap-southeast-2', service='EC2', wantlist=True) }}"
tasks:
- name: "use list return option and iterate as a loop"
debug: msg="{% for cidr in ec2_ranges %}{{ cidr }} {% endfor %}"
# "52.62.0.0/15 52.64.0.0/17 52.64.128.0/17 52.65.0.0/16 52.95.241.0/24 52.95.255.16/28 54.66.0.0/16 "
- name: "Pull S3 IP ranges, and print the default return style"
debug: msg="{{ lookup('aws_service_ip_ranges', region='us-east-1', service='S3') }}"
# "52.92.16.0/20,52.216.0.0/15,54.231.0.0/17"
"""
RETURN = """
_raw:
description: comma-separated list of CIDR ranges
"""
import json
from ansible.errors import AnsibleError
from ansible.module_utils.six.moves.urllib.error import HTTPError
from ansible.module_utils.six.moves.urllib.error import URLError
from ansible.module_utils._text import to_native
from ansible.module_utils.urls import ConnectionError
from ansible.module_utils.urls import open_url
from ansible.module_utils.urls import SSLValidationError
from ansible.plugins.lookup import LookupBase
class LookupModule(LookupBase):
def run(self, terms, variables, **kwargs):
if "ipv6_prefixes" in kwargs and kwargs["ipv6_prefixes"]:
prefixes_label = "ipv6_prefixes"
ip_prefix_label = "ipv6_prefix"
else:
prefixes_label = "prefixes"
ip_prefix_label = "ip_prefix"
try:
resp = open_url('https://ip-ranges.amazonaws.com/ip-ranges.json')
amazon_response = json.load(resp)[prefixes_label]
except getattr(json.decoder, 'JSONDecodeError', ValueError) as e:
# on Python 3+, json.decoder.JSONDecodeError is raised for bad
# JSON. On 2.x it's a ValueError
raise AnsibleError("Could not decode AWS IP ranges: %s" % to_native(e))
except HTTPError as e:
raise AnsibleError("Received HTTP error while pulling IP ranges: %s" % to_native(e))
except SSLValidationError as e:
raise AnsibleError("Error validating the server's certificate for: %s" % to_native(e))
except URLError as e:
raise AnsibleError("Failed look up IP range service: %s" % to_native(e))
except ConnectionError as e:
raise AnsibleError("Error connecting to IP range service: %s" % to_native(e))
if 'region' in kwargs:
region = kwargs['region']
amazon_response = (item for item in amazon_response if item['region'] == region)
if 'service' in kwargs:
service = str.upper(kwargs['service'])
amazon_response = (item for item in amazon_response if item['service'] == service)
iprange = [item[ip_prefix_label] for item in amazon_response]
return iprange

View File

@@ -0,0 +1,286 @@
# (c) 2016, Bill Wang <ozbillwang(at)gmail.com>
# (c) 2017, Marat Bakeev <hawara(at)gmail.com>
# (c) 2018, Michael De La Rue <siblemitcom.mddlr(at)spamgourmet.com>
# (c) 2017 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
DOCUMENTATION = '''
name: aws_ssm
author:
- Bill Wang (!UNKNOWN) <ozbillwang(at)gmail.com>
- Marat Bakeev (!UNKNOWN) <hawara(at)gmail.com>
- Michael De La Rue (!UNKNOWN) <siblemitcom.mddlr@spamgourmet.com>
short_description: Get the value for a SSM parameter or all parameters under a path
description:
- Get the value for an Amazon Simple Systems Manager parameter or a hierarchy of parameters.
The first argument you pass the lookup can either be a parameter name or a hierarchy of
parameters. Hierarchies start with a forward slash and end with the parameter name. Up to
5 layers may be specified.
- If looking up an explicitly listed parameter by name which does not exist then the lookup
will generate an error. You can use the ```default``` filter to give a default value in
this case but must set the ```on_missing``` parameter to ```skip``` or ```warn```. You must
also set the second parameter of the ```default``` filter to ```true``` (see examples below).
- When looking up a path for parameters under it a dictionary will be returned for each path.
If there is no parameter under that path then the lookup will generate an error.
- If the lookup fails due to lack of permissions or due to an AWS client error then the aws_ssm
will generate an error. If you want to continue in this case then you will have to set up
two ansible tasks, one which sets a variable and ignores failures and one which uses the value
of that variable with a default. See the examples below.
options:
decrypt:
description: A boolean to indicate whether to decrypt the parameter.
default: true
type: boolean
bypath:
description: A boolean to indicate whether the parameter is provided as a hierarchy.
default: false
type: boolean
recursive:
description: A boolean to indicate whether to retrieve all parameters within a hierarchy.
default: false
type: boolean
shortnames:
description: Indicates whether to return the name only without path if using a parameter hierarchy.
default: false
type: boolean
on_missing:
description:
- Action to take if the SSM parameter is missing.
- C(error) will raise a fatal error when the SSM parameter is missing.
- C(skip) will silently ignore the missing SSM parameter.
- C(warn) will skip over the missing SSM parameter but issue a warning.
default: error
type: string
choices: ['error', 'skip', 'warn']
version_added: 2.0.0
on_denied:
description:
- Action to take if access to the SSM parameter is denied.
- C(error) will raise a fatal error when access to the SSM parameter is denied.
- C(skip) will silently ignore the denied SSM parameter.
- C(warn) will skip over the denied SSM parameter but issue a warning.
default: error
type: string
choices: ['error', 'skip', 'warn']
version_added: 2.0.0
endpoint:
description: Use a custom endpoint when connecting to SSM service.
type: string
version_added: 3.3.0
extends_documentation_fragment:
- amazon.aws.boto3
'''
EXAMPLES = '''
# lookup sample:
- name: lookup ssm parameter store in the current region
debug: msg="{{ lookup('aws_ssm', 'Hello' ) }}"
- name: lookup ssm parameter store in specified region
debug: msg="{{ lookup('aws_ssm', 'Hello', region='us-east-2' ) }}"
- name: lookup ssm parameter store without decryption
debug: msg="{{ lookup('aws_ssm', 'Hello', decrypt=False ) }}"
- name: lookup ssm parameter store using a specified aws profile
debug: msg="{{ lookup('aws_ssm', 'Hello', aws_profile='myprofile' ) }}"
- name: lookup ssm parameter store using explicit aws credentials
debug: msg="{{ lookup('aws_ssm', 'Hello', aws_access_key=my_aws_access_key, aws_secret_key=my_aws_secret_key, aws_security_token=my_security_token ) }}"
- name: lookup ssm parameter store with all options
debug: msg="{{ lookup('aws_ssm', 'Hello', decrypt=false, region='us-east-2', aws_profile='myprofile') }}"
- name: lookup ssm parameter and fail if missing
debug: msg="{{ lookup('aws_ssm', 'missing-parameter') }}"
- name: lookup a key which doesn't exist, returning a default ('root')
debug: msg="{{ lookup('aws_ssm', 'AdminID', on_missing="skip") | default('root', true) }}"
- name: lookup a key which doesn't exist failing to store it in a fact
set_fact:
temp_secret: "{{ lookup('aws_ssm', '/NoAccess/hiddensecret') }}"
ignore_errors: true
- name: show fact default to "access failed" if we don't have access
debug: msg="{{ 'the secret was:' ~ temp_secret | default('could not access secret') }}"
- name: return a dictionary of ssm parameters from a hierarchy path
debug: msg="{{ lookup('aws_ssm', '/PATH/to/params', region='ap-southeast-2', bypath=true, recursive=true ) }}"
- name: return a dictionary of ssm parameters from a hierarchy path with shortened names (param instead of /PATH/to/param)
debug: msg="{{ lookup('aws_ssm', '/PATH/to/params', region='ap-southeast-2', shortnames=true, bypath=true, recursive=true ) }}"
- name: Iterate over a parameter hierarchy (one iteration per parameter)
debug: msg='Key contains {{ item.key }} , with value {{ item.value }}'
loop: '{{ lookup("aws_ssm", "/demo/", region="ap-southeast-2", bypath=True) | dict2items }}'
- name: Iterate over multiple paths as dictionaries (one iteration per path)
debug: msg='Path contains {{ item }}'
loop: '{{ lookup("aws_ssm", "/demo/", "/demo1/", bypath=True)}}'
- name: lookup ssm parameter warn if access is denied
debug: msg="{{ lookup('aws_ssm', 'missing-parameter', on_denied="warn" ) }}"
'''
try:
import botocore
except ImportError:
pass # will be captured by imported HAS_BOTO3
from ansible.errors import AnsibleError
from ansible.module_utils._text import to_native
from ansible.plugins.lookup import LookupBase
from ansible.utils.display import Display
from ansible.module_utils.six import string_types
from ansible.module_utils.basic import missing_required_lib
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import boto3_conn
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import get_aws_connection_info
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import HAS_BOTO3
from ansible_collections.amazon.aws.plugins.module_utils.ec2 import boto3_tag_list_to_ansible_dict
from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
display = Display()
class LookupModule(LookupBase):
def run(self, terms, variables=None, boto_profile=None, aws_profile=None,
aws_secret_key=None, aws_access_key=None, aws_security_token=None, region=None,
bypath=False, shortnames=False, recursive=False, decrypt=True, on_missing="error",
on_denied="error", endpoint=None):
'''
:arg terms: a list of lookups to run.
e.g. ['parameter_name', 'parameter_name_too' ]
:kwarg variables: ansible variables active at the time of the lookup
:kwarg aws_secret_key: identity of the AWS key to use
:kwarg aws_access_key: AWS secret key (matching identity)
:kwarg aws_security_token: AWS session key if using STS
:kwarg decrypt: Set to True to get decrypted parameters
:kwarg region: AWS region in which to do the lookup
:kwarg bypath: Set to True to do a lookup of variables under a path
:kwarg recursive: Set to True to recurse below the path (requires bypath=True)
:kwarg on_missing: Action to take if the SSM parameter is missing
:kwarg on_denied: Action to take if access to the SSM parameter is denied
:kwarg endpoint: Endpoint for SSM client
:returns: A list of parameter values or a list of dictionaries if bypath=True.
'''
if not HAS_BOTO3:
raise AnsibleError(missing_required_lib('botocore and boto3'))
# validate arguments 'on_missing' and 'on_denied'
if on_missing is not None and (not isinstance(on_missing, string_types) or on_missing.lower() not in ['error', 'warn', 'skip']):
raise AnsibleError('"on_missing" must be a string and one of "error", "warn" or "skip", not %s' % on_missing)
if on_denied is not None and (not isinstance(on_denied, string_types) or on_denied.lower() not in ['error', 'warn', 'skip']):
raise AnsibleError('"on_denied" must be a string and one of "error", "warn" or "skip", not %s' % on_denied)
ret = []
ssm_dict = {}
self.params = variables
cli_region, cli_endpoint, cli_boto_params = get_aws_connection_info(self, boto3=True)
if region:
cli_region = region
if endpoint:
cli_endpoint = endpoint
# For backward compatibility
if aws_access_key:
cli_boto_params.update({'aws_access_key_id': aws_access_key})
if aws_secret_key:
cli_boto_params.update({'aws_secret_access_key': aws_secret_key})
if aws_security_token:
cli_boto_params.update({'aws_session_token': aws_security_token})
if boto_profile:
cli_boto_params.update({'profile_name': boto_profile})
if aws_profile:
cli_boto_params.update({'profile_name': aws_profile})
cli_boto_params.update(dict(
conn_type='client',
resource='ssm',
region=cli_region,
endpoint=cli_endpoint,
))
client = boto3_conn(module=self, **cli_boto_params)
ssm_dict['WithDecryption'] = decrypt
# Lookup by path
if bypath:
ssm_dict['Recursive'] = recursive
for term in terms:
display.vvv("AWS_ssm path lookup term: %s in region: %s" % (term, region))
paramlist = self.get_path_parameters(client, ssm_dict, term, on_missing.lower(), on_denied.lower())
# Shorten parameter names. Yes, this will return
# duplicate names with different values.
if shortnames:
for x in paramlist:
x['Name'] = x['Name'][x['Name'].rfind('/') + 1:]
display.vvvv("AWS_ssm path lookup returned: %s" % str(paramlist))
ret.append(boto3_tag_list_to_ansible_dict(paramlist,
tag_name_key_name="Name",
tag_value_key_name="Value"))
# Lookup by parameter name - always returns a list with one or
# no entry.
else:
display.vvv("AWS_ssm name lookup term: %s" % terms)
for term in terms:
ret.append(self.get_parameter_value(client, ssm_dict, term, on_missing.lower(), on_denied.lower()))
display.vvvv("AWS_ssm path lookup returning: %s " % str(ret))
return ret
def get_path_parameters(self, client, ssm_dict, term, on_missing, on_denied):
ssm_dict["Path"] = term
paginator = client.get_paginator('get_parameters_by_path')
try:
paramlist = paginator.paginate(**ssm_dict).build_full_result()['Parameters']
except is_boto3_error_code('AccessDeniedException'):
if on_denied == 'error':
raise AnsibleError("Failed to access SSM parameter path %s (AccessDenied)" % term)
elif on_denied == 'warn':
self._display.warning('Skipping, access denied for SSM parameter path %s' % term)
paramlist = [{}]
elif on_denied == 'skip':
paramlist = [{}]
except botocore.exceptions.ClientError as e: # pylint: disable=duplicate-except
raise AnsibleError("SSM lookup exception: {0}".format(to_native(e)))
if not len(paramlist):
if on_missing == "error":
raise AnsibleError("Failed to find SSM parameter path %s (ResourceNotFound)" % term)
elif on_missing == "warn":
self._display.warning('Skipping, did not find SSM parameter path %s' % term)
return paramlist
def get_parameter_value(self, client, ssm_dict, term, on_missing, on_denied):
ssm_dict["Name"] = term
try:
response = client.get_parameter(**ssm_dict)
return response['Parameter']['Value']
except is_boto3_error_code('ParameterNotFound'):
if on_missing == 'error':
raise AnsibleError("Failed to find SSM parameter %s (ResourceNotFound)" % term)
elif on_missing == 'warn':
self._display.warning('Skipping, did not find SSM parameter %s' % term)
except is_boto3_error_code('AccessDeniedException'): # pylint: disable=duplicate-except
if on_denied == 'error':
raise AnsibleError("Failed to access SSM parameter %s (AccessDenied)" % term)
elif on_denied == 'warn':
self._display.warning('Skipping, access denied for SSM parameter %s' % term)
except botocore.exceptions.ClientError as e: # pylint: disable=duplicate-except
raise AnsibleError("SSM lookup exception: {0}".format(to_native(e)))
return None

View File

@@ -0,0 +1,344 @@
# Vendored copy of distutils/version.py from CPython 3.9.5
#
# Implements multiple version numbering conventions for the
# Python Module Distribution Utilities.
#
# PSF License (see PSF-license.txt or https://opensource.org/licenses/Python-2.0)
#
"""Provides classes to represent module version numbers (one class for
each style of version numbering). There are currently two such classes
implemented: StrictVersion and LooseVersion.
Every version number class implements the following interface:
* the 'parse' method takes a string and parses it to some internal
representation; if the string is an invalid version number,
'parse' raises a ValueError exception
* the class constructor takes an optional string argument which,
if supplied, is passed to 'parse'
* __str__ reconstructs the string that was passed to 'parse' (or
an equivalent string -- ie. one that will generate an equivalent
version number instance)
* __repr__ generates Python code to recreate the version number instance
* _cmp compares the current instance with either another instance
of the same class or a string (which will be parsed to an instance
of the same class, thus must follow the same rules)
"""
from __future__ import absolute_import, division, print_function
__metaclass__ = type
import re
try:
RE_FLAGS = re.VERBOSE | re.ASCII
except AttributeError:
RE_FLAGS = re.VERBOSE
class Version:
"""Abstract base class for version numbering classes. Just provides
constructor (__init__) and reproducer (__repr__), because those
seem to be the same for all version numbering classes; and route
rich comparisons to _cmp.
"""
def __init__(self, vstring=None):
if vstring:
self.parse(vstring)
def __repr__(self):
return "%s ('%s')" % (self.__class__.__name__, str(self))
def __eq__(self, other):
c = self._cmp(other)
if c is NotImplemented:
return c
return c == 0
def __lt__(self, other):
c = self._cmp(other)
if c is NotImplemented:
return c
return c < 0
def __le__(self, other):
c = self._cmp(other)
if c is NotImplemented:
return c
return c <= 0
def __gt__(self, other):
c = self._cmp(other)
if c is NotImplemented:
return c
return c > 0
def __ge__(self, other):
c = self._cmp(other)
if c is NotImplemented:
return c
return c >= 0
# Interface for version-number classes -- must be implemented
# by the following classes (the concrete ones -- Version should
# be treated as an abstract class).
# __init__ (string) - create and take same action as 'parse'
# (string parameter is optional)
# parse (string) - convert a string representation to whatever
# internal representation is appropriate for
# this style of version numbering
# __str__ (self) - convert back to a string; should be very similar
# (if not identical to) the string supplied to parse
# __repr__ (self) - generate Python code to recreate
# the instance
# _cmp (self, other) - compare two version numbers ('other' may
# be an unparsed version string, or another
# instance of your version class)
class StrictVersion(Version):
"""Version numbering for anal retentives and software idealists.
Implements the standard interface for version number classes as
described above. A version number consists of two or three
dot-separated numeric components, with an optional "pre-release" tag
on the end. The pre-release tag consists of the letter 'a' or 'b'
followed by a number. If the numeric components of two version
numbers are equal, then one with a pre-release tag will always
be deemed earlier (lesser) than one without.
The following are valid version numbers (shown in the order that
would be obtained by sorting according to the supplied cmp function):
0.4 0.4.0 (these two are equivalent)
0.4.1
0.5a1
0.5b3
0.5
0.9.6
1.0
1.0.4a3
1.0.4b1
1.0.4
The following are examples of invalid version numbers:
1
2.7.2.2
1.3.a4
1.3pl1
1.3c4
The rationale for this version numbering system will be explained
in the distutils documentation.
"""
version_re = re.compile(r"^(\d+) \. (\d+) (\. (\d+))? ([ab](\d+))?$", RE_FLAGS)
def parse(self, vstring):
match = self.version_re.match(vstring)
if not match:
raise ValueError("invalid version number '%s'" % vstring)
(major, minor, patch, prerelease, prerelease_num) = match.group(1, 2, 4, 5, 6)
if patch:
self.version = tuple(map(int, [major, minor, patch]))
else:
self.version = tuple(map(int, [major, minor])) + (0,)
if prerelease:
self.prerelease = (prerelease[0], int(prerelease_num))
else:
self.prerelease = None
def __str__(self):
if self.version[2] == 0:
vstring = ".".join(map(str, self.version[0:2]))
else:
vstring = ".".join(map(str, self.version))
if self.prerelease:
vstring = vstring + self.prerelease[0] + str(self.prerelease[1])
return vstring
def _cmp(self, other):
if isinstance(other, str):
other = StrictVersion(other)
elif not isinstance(other, StrictVersion):
return NotImplemented
if self.version != other.version:
# numeric versions don't match
# prerelease stuff doesn't matter
if self.version < other.version:
return -1
else:
return 1
# have to compare prerelease
# case 1: neither has prerelease; they're equal
# case 2: self has prerelease, other doesn't; other is greater
# case 3: self doesn't have prerelease, other does: self is greater
# case 4: both have prerelease: must compare them!
if not self.prerelease and not other.prerelease:
return 0
elif self.prerelease and not other.prerelease:
return -1
elif not self.prerelease and other.prerelease:
return 1
elif self.prerelease and other.prerelease:
if self.prerelease == other.prerelease:
return 0
elif self.prerelease < other.prerelease:
return -1
else:
return 1
else:
raise AssertionError("never get here")
# end class StrictVersion
# The rules according to Greg Stein:
# 1) a version number has 1 or more numbers separated by a period or by
# sequences of letters. If only periods, then these are compared
# left-to-right to determine an ordering.
# 2) sequences of letters are part of the tuple for comparison and are
# compared lexicographically
# 3) recognize the numeric components may have leading zeroes
#
# The LooseVersion class below implements these rules: a version number
# string is split up into a tuple of integer and string components, and
# comparison is a simple tuple comparison. This means that version
# numbers behave in a predictable and obvious way, but a way that might
# not necessarily be how people *want* version numbers to behave. There
# wouldn't be a problem if people could stick to purely numeric version
# numbers: just split on period and compare the numbers as tuples.
# However, people insist on putting letters into their version numbers;
# the most common purpose seems to be:
# - indicating a "pre-release" version
# ('alpha', 'beta', 'a', 'b', 'pre', 'p')
# - indicating a post-release patch ('p', 'pl', 'patch')
# but of course this can't cover all version number schemes, and there's
# no way to know what a programmer means without asking him.
#
# The problem is what to do with letters (and other non-numeric
# characters) in a version number. The current implementation does the
# obvious and predictable thing: keep them as strings and compare
# lexically within a tuple comparison. This has the desired effect if
# an appended letter sequence implies something "post-release":
# eg. "0.99" < "0.99pl14" < "1.0", and "5.001" < "5.001m" < "5.002".
#
# However, if letters in a version number imply a pre-release version,
# the "obvious" thing isn't correct. Eg. you would expect that
# "1.5.1" < "1.5.2a2" < "1.5.2", but under the tuple/lexical comparison
# implemented here, this just isn't so.
#
# Two possible solutions come to mind. The first is to tie the
# comparison algorithm to a particular set of semantic rules, as has
# been done in the StrictVersion class above. This works great as long
# as everyone can go along with bondage and discipline. Hopefully a
# (large) subset of Python module programmers will agree that the
# particular flavour of bondage and discipline provided by StrictVersion
# provides enough benefit to be worth using, and will submit their
# version numbering scheme to its domination. The free-thinking
# anarchists in the lot will never give in, though, and something needs
# to be done to accommodate them.
#
# Perhaps a "moderately strict" version class could be implemented that
# lets almost anything slide (syntactically), and makes some heuristic
# assumptions about non-digits in version number strings. This could
# sink into special-case-hell, though; if I was as talented and
# idiosyncratic as Larry Wall, I'd go ahead and implement a class that
# somehow knows that "1.2.1" < "1.2.2a2" < "1.2.2" < "1.2.2pl3", and is
# just as happy dealing with things like "2g6" and "1.13++". I don't
# think I'm smart enough to do it right though.
#
# In any case, I've coded the test suite for this module (see
# ../test/test_version.py) specifically to fail on things like comparing
# "1.2a2" and "1.2". That's not because the *code* is doing anything
# wrong, it's because the simple, obvious design doesn't match my
# complicated, hairy expectations for real-world version numbers. It
# would be a snap to fix the test suite to say, "Yep, LooseVersion does
# the Right Thing" (ie. the code matches the conception). But I'd rather
# have a conception that matches common notions about version numbers.
class LooseVersion(Version):
"""Version numbering for anarchists and software realists.
Implements the standard interface for version number classes as
described above. A version number consists of a series of numbers,
separated by either periods or strings of letters. When comparing
version numbers, the numeric components will be compared
numerically, and the alphabetic components lexically. The following
are all valid version numbers, in no particular order:
1.5.1
1.5.2b2
161
3.10a
8.02
3.4j
1996.07.12
3.2.pl0
3.1.1.6
2g6
11g
0.960923
2.2beta29
1.13++
5.5.kw
2.0b1pl0
In fact, there is no such thing as an invalid version number under
this scheme; the rules for comparison are simple and predictable,
but may not always give the results you want (for some definition
of "want").
"""
component_re = re.compile(r"(\d+ | [a-z]+ | \.)", re.VERBOSE)
def __init__(self, vstring=None):
if vstring:
self.parse(vstring)
def parse(self, vstring):
# I've given up on thinking I can reconstruct the version string
# from the parsed tuple -- so I just store the string here for
# use by __str__
self.vstring = vstring
components = [x for x in self.component_re.split(vstring) if x and x != "."]
for i, obj in enumerate(components):
try:
components[i] = int(obj)
except ValueError:
pass
self.version = components
def __str__(self):
return self.vstring
def __repr__(self):
return "LooseVersion ('%s')" % str(self)
def _cmp(self, other):
if isinstance(other, str):
other = LooseVersion(other)
elif not isinstance(other, LooseVersion):
return NotImplemented
if self.version == other.version:
return 0
if self.version < other.version:
return -1
if self.version > other.version:
return 1
# end class LooseVersion

View File

@@ -0,0 +1,222 @@
# -*- coding: utf-8 -*-
#
# Copyright (c) 2019 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
#
# This module is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This software is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this software. If not, see <http://www.gnu.org/licenses/>.
#
# Author:
# - Matthew Davis <Matthew.Davis.2@team.telstra.com>
# on behalf of Telstra Corporation Limited
#
# Common functionality to be used by the modules:
# - acm_certificate
# - acm_certificate_info
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
"""
Common Amazon Certificate Manager facts shared between modules
"""
try:
from botocore.exceptions import BotoCoreError, ClientError
except ImportError:
pass
from ansible.module_utils._text import to_bytes
from ansible.module_utils.common.dict_transformations import camel_dict_to_snake_dict
from .core import is_boto3_error_code
from .ec2 import AWSRetry
from .ec2 import ansible_dict_to_boto3_tag_list
from .ec2 import boto3_tag_list_to_ansible_dict
class ACMServiceManager(object):
"""Handles ACM Facts Services"""
def __init__(self, module):
self.module = module
self.client = module.client('acm')
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException'])
def delete_certificate_with_backoff(self, client, arn):
client.delete_certificate(CertificateArn=arn)
def delete_certificate(self, client, module, arn):
module.debug("Attempting to delete certificate %s" % arn)
try:
self.delete_certificate_with_backoff(client, arn)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Couldn't delete certificate %s" % arn)
module.debug("Successfully deleted certificate %s" % arn)
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException'])
def list_certificates_with_backoff(self, client, statuses=None):
paginator = client.get_paginator('list_certificates')
kwargs = dict()
if statuses:
kwargs['CertificateStatuses'] = statuses
return paginator.paginate(**kwargs).build_full_result()['CertificateSummaryList']
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException', 'ResourceNotFoundException'])
def get_certificate_with_backoff(self, client, certificate_arn):
response = client.get_certificate(CertificateArn=certificate_arn)
# strip out response metadata
return {'Certificate': response['Certificate'],
'CertificateChain': response['CertificateChain']}
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException', 'ResourceNotFoundException'])
def describe_certificate_with_backoff(self, client, certificate_arn):
return client.describe_certificate(CertificateArn=certificate_arn)['Certificate']
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException', 'ResourceNotFoundException'])
def list_certificate_tags_with_backoff(self, client, certificate_arn):
return client.list_tags_for_certificate(CertificateArn=certificate_arn)['Tags']
# Returns a list of certificates
# if domain_name is specified, returns only certificates with that domain
# if an ARN is specified, returns only that certificate
# only_tags is a dict, e.g. {'key':'value'}. If specified this function will return
# only certificates which contain all those tags (key exists, value matches).
def get_certificates(self, client, module, domain_name=None, statuses=None, arn=None, only_tags=None):
try:
all_certificates = self.list_certificates_with_backoff(client=client, statuses=statuses)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Couldn't obtain certificates")
if domain_name:
certificates = [cert for cert in all_certificates
if cert['DomainName'] == domain_name]
else:
certificates = all_certificates
if arn:
# still return a list, not just one item
certificates = [c for c in certificates if c['CertificateArn'] == arn]
results = []
for certificate in certificates:
try:
cert_data = self.describe_certificate_with_backoff(client, certificate['CertificateArn'])
except is_boto3_error_code('ResourceNotFoundException'):
# The certificate was deleted after the call to list_certificates_with_backoff.
continue
except (BotoCoreError, ClientError) as e: # pylint: disable=duplicate-except
module.fail_json_aws(e, msg="Couldn't obtain certificate metadata for domain %s" % certificate['DomainName'])
# in some states, ACM resources do not have a corresponding cert
if cert_data['Status'] not in ['PENDING_VALIDATION', 'VALIDATION_TIMED_OUT', 'FAILED']:
try:
cert_data.update(self.get_certificate_with_backoff(client, certificate['CertificateArn']))
except is_boto3_error_code('ResourceNotFoundException'):
# The certificate was deleted after the call to list_certificates_with_backoff.
continue
except (BotoCoreError, ClientError, KeyError) as e: # pylint: disable=duplicate-except
module.fail_json_aws(e, msg="Couldn't obtain certificate data for domain %s" % certificate['DomainName'])
cert_data = camel_dict_to_snake_dict(cert_data)
try:
tags = self.list_certificate_tags_with_backoff(client, certificate['CertificateArn'])
except is_boto3_error_code('ResourceNotFoundException'):
# The certificate was deleted after the call to list_certificates_with_backoff.
continue
except (BotoCoreError, ClientError) as e: # pylint: disable=duplicate-except
module.fail_json_aws(e, msg="Couldn't obtain tags for domain %s" % certificate['DomainName'])
cert_data['tags'] = boto3_tag_list_to_ansible_dict(tags)
results.append(cert_data)
if only_tags:
for tag_key in only_tags:
try:
results = [c for c in results if ('tags' in c) and (tag_key in c['tags']) and (c['tags'][tag_key] == only_tags[tag_key])]
except (TypeError, AttributeError) as e:
for c in results:
if 'tags' not in c:
module.debug("cert is %s" % str(c))
module.fail_json(msg="ACM tag filtering err", exception=e)
return results
# returns the domain name of a certificate (encoded in the public cert)
# for a given ARN
# A cert with that ARN must already exist
def get_domain_of_cert(self, client, module, arn):
if arn is None:
module.fail(msg="Internal error with ACM domain fetching, no certificate ARN specified")
try:
cert_data = self.describe_certificate_with_backoff(client=client, certificate_arn=arn)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Couldn't obtain certificate data for arn %s" % arn)
return cert_data['DomainName']
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException'])
def import_certificate_with_backoff(self, client, certificate, private_key, certificate_chain, arn):
if certificate_chain:
if arn:
ret = client.import_certificate(Certificate=to_bytes(certificate),
PrivateKey=to_bytes(private_key),
CertificateChain=to_bytes(certificate_chain),
CertificateArn=arn)
else:
ret = client.import_certificate(Certificate=to_bytes(certificate),
PrivateKey=to_bytes(private_key),
CertificateChain=to_bytes(certificate_chain))
else:
if arn:
ret = client.import_certificate(Certificate=to_bytes(certificate),
PrivateKey=to_bytes(private_key),
CertificateArn=arn)
else:
ret = client.import_certificate(Certificate=to_bytes(certificate),
PrivateKey=to_bytes(private_key))
return ret['CertificateArn']
# Tags are a normal Ansible style dict
# {'Key':'Value'}
@AWSRetry.jittered_backoff(delay=5, catch_extra_error_codes=['RequestInProgressException', 'ResourceNotFoundException'])
def tag_certificate_with_backoff(self, client, arn, tags):
aws_tags = ansible_dict_to_boto3_tag_list(tags)
client.add_tags_to_certificate(CertificateArn=arn, Tags=aws_tags)
def import_certificate(self, client, module, certificate, private_key, arn=None, certificate_chain=None, tags=None):
original_arn = arn
# upload cert
try:
arn = self.import_certificate_with_backoff(client, certificate, private_key, certificate_chain, arn)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Couldn't upload new certificate")
if original_arn and (arn != original_arn):
# I'm not sure whether the API guarentees that the ARN will not change
# I'm failing just in case.
# If I'm wrong, I'll catch it in the integration tests.
module.fail_json(msg="ARN changed with ACM update, from %s to %s" % (original_arn, arn))
# tag that cert
try:
self.tag_certificate_with_backoff(client, arn, tags)
except (BotoCoreError, ClientError) as e:
module.debug("Attempting to delete the cert we just created, arn=%s" % arn)
try:
self.delete_certificate_with_backoff(client, arn)
except (BotoCoreError, ClientError):
module.warn("Certificate %s exists, and is not tagged. So Ansible will not see it on the next run.")
module.fail_json_aws(e, msg="Couldn't tag certificate %s, couldn't delete it either" % arn)
module.fail_json_aws(e, msg="Couldn't tag certificate %s" % arn)
return arn

View File

@@ -0,0 +1,69 @@
#
# Copyright 2017 Michael De La Rue | Ansible
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import re
def parse_aws_arn(arn):
"""
The following are the general formats for ARNs.
arn:partition:service:region:account-id:resource-id
arn:partition:service:region:account-id:resource-type/resource-id
arn:partition:service:region:account-id:resource-type:resource-id
The specific formats depend on the resource.
The ARNs for some resources omit the Region, the account ID, or both the Region and the account ID.
"""
m = re.search(r"arn:(aws(-([a-z\-]+))?):([\w-]+):([a-z0-9\-]*):(\d*|aws|aws-managed):(.*)", arn)
if m is None:
return None
result = dict()
result.update(dict(partition=m.group(1)))
result.update(dict(service=m.group(4)))
result.update(dict(region=m.group(5)))
result.update(dict(account_id=m.group(6)))
result.update(dict(resource=m.group(7)))
return result
# An implementation of this used was originally in ec2.py, however Outposts
# aren't specific to the EC2 service
def is_outpost_arn(arn):
"""
Validates that the ARN is for an AWS Outpost
API Specification Document:
https://docs.aws.amazon.com/outposts/latest/APIReference/API_Outpost.html
"""
details = parse_aws_arn(arn)
if not details:
return False
service = details.get('service') or ""
if service.lower() != 'outposts':
return False
resource = details.get('resource') or ""
if not re.match('^outpost/op-[a-f0-9]{17}$', resource):
return False
return True

View File

@@ -0,0 +1,58 @@
# Copyright (c) 2017 Ansible Project
#
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
"""
This module adds shared support for Batch modules.
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from ansible.module_utils.common.dict_transformations import snake_dict_to_camel_dict
def cc(key):
"""
Changes python key into Camel case equivalent. For example, 'compute_environment_name' becomes
'computeEnvironmentName'.
:param key:
:return:
"""
components = key.split('_')
return components[0] + "".join([token.capitalize() for token in components[1:]])
def set_api_params(module, module_params):
"""
Sets module parameters to those expected by the boto3 API.
:param module:
:param module_params:
:return:
"""
api_params = dict((k, v) for k, v in dict(module.params).items() if k in module_params and v is not None)
return snake_dict_to_camel_dict(api_params)

View File

@@ -0,0 +1,357 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Michael DeHaan <michael.dehaan@gmail.com>, 2012-2013
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
A set of helper functions designed to help with initializing boto3/botocore
connections.
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import json
import os
import traceback
BOTO3_IMP_ERR = None
try:
import boto3
import botocore
HAS_BOTO3 = True
except ImportError:
BOTO3_IMP_ERR = traceback.format_exc()
HAS_BOTO3 = False
from ansible.module_utils._text import to_native
from ansible.module_utils.ansible_release import __version__
from ansible.module_utils.basic import missing_required_lib
from ansible.module_utils.six import binary_type
from ansible.module_utils.six import text_type
from .retries import AWSRetry
def boto3_conn(module, conn_type=None, resource=None, region=None, endpoint=None, **params):
"""
Builds a boto3 resource/client connection cleanly wrapping the most common failures.
Handles:
ValueError,
botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError,
botocore.exceptions.NoCredentialsError, botocore.exceptions.ConfigParseError,
botocore.exceptions.NoRegionError
"""
try:
return _boto3_conn(conn_type=conn_type, resource=resource, region=region, endpoint=endpoint, **params)
except ValueError as e:
module.fail_json(msg="Couldn't connect to AWS: %s" % to_native(e))
except (botocore.exceptions.ProfileNotFound, botocore.exceptions.PartialCredentialsError,
botocore.exceptions.NoCredentialsError, botocore.exceptions.ConfigParseError) as e:
module.fail_json(msg=to_native(e))
except botocore.exceptions.NoRegionError:
module.fail_json(msg="The %s module requires a region and none was found in configuration, "
"environment variables or module parameters" % module._name)
def _boto3_conn(conn_type=None, resource=None, region=None, endpoint=None, **params):
"""
Builds a boto3 resource/client connection cleanly wrapping the most common failures.
No exceptions are caught/handled.
"""
profile = params.pop('profile_name', None)
if conn_type not in ['both', 'resource', 'client']:
raise ValueError('There is an issue in the calling code. You '
'must specify either both, resource, or client to '
'the conn_type parameter in the boto3_conn function '
'call')
config = botocore.config.Config(
user_agent_extra='Ansible/{0}'.format(__version__),
)
if params.get('config') is not None:
config = config.merge(params.pop('config'))
if params.get('aws_config') is not None:
config = config.merge(params.pop('aws_config'))
session = boto3.session.Session(
profile_name=profile,
)
enable_placebo(session)
if conn_type == 'resource':
return session.resource(resource, config=config, region_name=region, endpoint_url=endpoint, **params)
elif conn_type == 'client':
return session.client(resource, config=config, region_name=region, endpoint_url=endpoint, **params)
else:
client = session.client(resource, region_name=region, endpoint_url=endpoint, **params)
resource = session.resource(resource, region_name=region, endpoint_url=endpoint, **params)
return client, resource
# Inventory plugins don't have access to the same 'module', they need to throw
# an exception rather than calling module.fail_json
boto3_inventory_conn = _boto3_conn
def boto_exception(err):
"""
Extracts the error message from a boto exception.
:param err: Exception from boto
:return: Error message
"""
if hasattr(err, 'error_message'):
error = err.error_message
elif hasattr(err, 'message'):
error = str(err.message) + ' ' + str(err) + ' - ' + str(type(err))
else:
error = '%s: %s' % (Exception, err)
return error
def get_aws_region(module, boto3=None):
region = module.params.get('region')
if region:
return region
if not HAS_BOTO3:
module.fail_json(msg=missing_required_lib('boto3'), exception=BOTO3_IMP_ERR)
# here we don't need to make an additional call, will default to 'us-east-1' if the below evaluates to None.
try:
# Botocore doesn't like empty strings, make sure we default to None in the case of an empty
# string.
profile_name = module.params.get('profile') or None
return botocore.session.Session(profile=profile_name).get_config_variable('region')
except botocore.exceptions.ProfileNotFound:
return None
def get_aws_connection_info(module, boto3=None):
# Check module args for credentials, then check environment vars
# access_key
endpoint_url = module.params.get('endpoint_url')
access_key = module.params.get('access_key')
secret_key = module.params.get('secret_key')
session_token = module.params.get('session_token')
region = get_aws_region(module)
profile_name = module.params.get('profile')
validate_certs = module.params.get('validate_certs')
ca_bundle = module.params.get('aws_ca_bundle')
config = module.params.get('aws_config')
if profile_name and (access_key or secret_key or session_token):
module.fail_json(msg="Passing both a profile and access tokens is not supported.")
# Botocore doesn't like empty strings, make sure we default to None in the case of an empty
# string.
if not access_key:
access_key = None
if not secret_key:
secret_key = None
if not session_token:
session_token = None
if profile_name:
boto_params = dict(
aws_access_key_id=None,
aws_secret_access_key=None,
aws_session_token=None,
profile_name=profile_name,
)
else:
boto_params = dict(
aws_access_key_id=access_key,
aws_secret_access_key=secret_key,
aws_session_token=session_token,
)
if validate_certs and ca_bundle:
boto_params['verify'] = ca_bundle
else:
boto_params['verify'] = validate_certs
if config is not None:
boto_params['aws_config'] = botocore.config.Config(**config)
for param, value in boto_params.items():
if isinstance(value, binary_type):
boto_params[param] = text_type(value, 'utf-8', 'strict')
return region, endpoint_url, boto_params
def _paginated_query(client, paginator_name, **params):
paginator = client.get_paginator(paginator_name)
result = paginator.paginate(**params).build_full_result()
return result
def paginated_query_with_retries(client, paginator_name, retry_decorator=None, **params):
"""
Performs a boto3 paginated query.
By default uses uses AWSRetry.jittered_backoff(retries=10) to retry queries
with temporary failures.
Examples:
tags = paginated_query_with_retries(client, "describe_tags", Filters=[])
decorator = AWSRetry.backoff(tries=5, delay=5, backoff=2.0,
catch_extra_error_codes=['RequestInProgressException'])
certificates = paginated_query_with_retries(client, "list_certificates", decorator)
"""
if retry_decorator is None:
retry_decorator = AWSRetry.jittered_backoff(retries=10)
result = retry_decorator(_paginated_query)(client, paginator_name, **params)
return result
def gather_sdk_versions():
"""Gather AWS SDK (boto3 and botocore) dependency versions
Returns {'boto3_version': str, 'botocore_version': str}
Returns {} if either module is not installed
"""
if not HAS_BOTO3:
return {}
import boto3
import botocore
return dict(boto3_version=boto3.__version__,
botocore_version=botocore.__version__)
def is_boto3_error_code(code, e=None):
"""Check if the botocore exception is raised by a specific error code.
Returns ClientError if the error code matches, a dummy exception if it does not have an error code or does not match
Example:
try:
ec2.describe_instances(InstanceIds=['potato'])
except is_boto3_error_code('InvalidInstanceID.Malformed'):
# handle the error for that code case
except botocore.exceptions.ClientError as e:
# handle the generic error case for all other codes
"""
from botocore.exceptions import ClientError
if e is None:
import sys
dummy, e, dummy = sys.exc_info()
if not isinstance(code, list):
code = [code]
if isinstance(e, ClientError) and e.response['Error']['Code'] in code:
return ClientError
return type('NeverEverRaisedException', (Exception,), {})
def is_boto3_error_message(msg, e=None):
"""Check if the botocore exception contains a specific error message.
Returns ClientError if the error code matches, a dummy exception if it does not have an error code or does not match
Example:
try:
ec2.describe_vpc_classic_link(VpcIds=[vpc_id])
except is_boto3_error_message('The functionality you requested is not available in this region.'):
# handle the error for that error message
except botocore.exceptions.ClientError as e:
# handle the generic error case for all other codes
"""
from botocore.exceptions import ClientError
if e is None:
import sys
dummy, e, dummy = sys.exc_info()
if isinstance(e, ClientError) and msg in e.response['Error']['Message']:
return ClientError
return type('NeverEverRaisedException', (Exception,), {})
def get_boto3_client_method_parameters(client, method_name, required=False):
op = client.meta.method_to_api_mapping.get(method_name)
input_shape = client._service_model.operation_model(op).input_shape
if not input_shape:
parameters = []
elif required:
parameters = list(input_shape.required_members)
else:
parameters = list(input_shape.members.keys())
return parameters
# Used by normalize_boto3_result
def _boto3_handler(obj):
if hasattr(obj, 'isoformat'):
return obj.isoformat()
else:
return obj
def normalize_boto3_result(result):
"""
Because Boto3 returns datetime objects where it knows things are supposed to
be dates we need to mass-convert them over to strings which Ansible/Jinja
handle better. This also makes it easier to compare complex objects which
include a mix of dates in string format (from parameters) and dates as
datetime objects. Boto3 is happy to be passed ISO8601 format strings.
"""
return json.loads(json.dumps(result, default=_boto3_handler))
def enable_placebo(session):
"""
Helper to record or replay offline modules for testing purpose.
"""
if "_ANSIBLE_PLACEBO_RECORD" in os.environ:
import placebo
existing_entries = os.listdir(os.environ["_ANSIBLE_PLACEBO_RECORD"])
idx = len(existing_entries)
data_path = f"{os.environ['_ANSIBLE_PLACEBO_RECORD']}/{idx}"
os.mkdir(data_path)
pill = placebo.attach(session, data_path=data_path)
pill.record()
if "_ANSIBLE_PLACEBO_REPLAY" in os.environ:
import shutil
import placebo
existing_entries = sorted([int(i) for i in os.listdir(os.environ["_ANSIBLE_PLACEBO_REPLAY"])])
idx = str(existing_entries[0])
data_path = os.environ['_ANSIBLE_PLACEBO_REPLAY'] + "/" + idx
try:
shutil.rmtree("_tmp")
except FileNotFoundError:
pass
shutil.move(data_path, "_tmp")
if len(existing_entries) == 1:
os.rmdir(os.environ["_ANSIBLE_PLACEBO_REPLAY"])
pill = placebo.attach(session, data_path="_tmp")
pill.playback()

View File

@@ -0,0 +1,213 @@
# Copyright (c) 2021 Ansible Project
#
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import time
import functools
import random
import ansible.module_utils.common.warnings as ansible_warnings
class BackoffIterator:
"""iterate sleep value based on the exponential or jitter back-off algorithm.
Args:
delay (int or float): initial delay.
backoff (int or float): backoff multiplier e.g. value of 2 will double the delay each retry.
max_delay (int or None): maximum amount of time to wait between retries.
jitter (bool): if set to true, add jitter to the generate value.
"""
def __init__(self, delay, backoff, max_delay=None, jitter=False):
self.delay = delay
self.backoff = backoff
self.max_delay = max_delay
self.jitter = jitter
def __iter__(self):
self.current_delay = self.delay
return self
def __next__(self):
return_value = self.current_delay if self.max_delay is None else min(self.current_delay, self.max_delay)
if self.jitter:
return_value = random.uniform(0.0, return_value)
self.current_delay *= self.backoff
return return_value
def _retry_func(func, sleep_time_generator, retries, catch_extra_error_codes, found_f, status_code_from_except_f, base_class):
counter = 0
for sleep_time in sleep_time_generator:
try:
return func()
except Exception as exc: # pylint: disable=broad-except
counter += 1
if counter == retries:
raise
if base_class and not isinstance(exc, base_class):
raise
status_code = status_code_from_except_f(exc)
if found_f(status_code, catch_extra_error_codes):
time.sleep(sleep_time)
else:
raise
class CloudRetry:
"""
The base class to be used by other cloud providers to provide a backoff/retry decorator based on status codes.
"""
base_class = type(None)
@staticmethod
def status_code_from_exception(error):
"""
Returns the Error 'code' from an exception.
Args:
error: The Exception from which the error code is to be extracted.
error will be an instance of class.base_class.
"""
raise NotImplementedError()
@staticmethod
def found(response_code, catch_extra_error_codes=None):
def _is_iterable():
try:
iter(catch_extra_error_codes)
except TypeError:
# not iterable
return False
else:
# iterable
return True
return _is_iterable() and response_code in catch_extra_error_codes
@classmethod
def base_decorator(cls, retries, found, status_code_from_exception, catch_extra_error_codes, sleep_time_generator):
def retry_decorator(func):
@functools.wraps(func)
def _retry_wrapper(*args, **kwargs):
partial_func = functools.partial(func, *args, **kwargs)
return _retry_func(
func=partial_func,
sleep_time_generator=sleep_time_generator,
retries=retries,
catch_extra_error_codes=catch_extra_error_codes,
found_f=found,
status_code_from_except_f=status_code_from_exception,
base_class=cls.base_class,
)
return _retry_wrapper
return retry_decorator
@classmethod
def exponential_backoff(cls, retries=10, delay=3, backoff=2, max_delay=60, catch_extra_error_codes=None):
"""Wrap a callable with retry behavior.
Args:
retries (int): Number of times to retry a failed request before giving up
default=10
delay (int or float): Initial delay between retries in seconds
default=3
backoff (int or float): backoff multiplier e.g. value of 2 will double the delay each retry
default=2
max_delay (int or None): maximum amount of time to wait between retries.
default=60
catch_extra_error_codes: Additional error messages to catch, in addition to those which may be defined by a subclass of CloudRetry
default=None
Returns:
Callable: A generator that calls the decorated function using an exponential backoff.
"""
sleep_time_generator = BackoffIterator(delay=delay, backoff=backoff, max_delay=max_delay)
return cls.base_decorator(
retries=retries,
found=cls.found,
status_code_from_exception=cls.status_code_from_exception,
catch_extra_error_codes=catch_extra_error_codes,
sleep_time_generator=sleep_time_generator,
)
@classmethod
def jittered_backoff(cls, retries=10, delay=3, backoff=2.0, max_delay=60, catch_extra_error_codes=None):
"""Wrap a callable with retry behavior.
Args:
retries (int): Number of times to retry a failed request before giving up
default=10
delay (int or float): Initial delay between retries in seconds
default=3
backoff (int or float): backoff multiplier e.g. value of 2 will double the delay each retry
default=2.0
max_delay (int or None): maximum amount of time to wait between retries.
default=60
catch_extra_error_codes: Additional error messages to catch, in addition to those which may be defined by a subclass of CloudRetry
default=None
Returns:
Callable: A generator that calls the decorated function using using a jittered backoff strategy.
"""
sleep_time_generator = BackoffIterator(delay=delay, backoff=backoff, max_delay=max_delay, jitter=True)
return cls.base_decorator(
retries=retries,
found=cls.found,
status_code_from_exception=cls.status_code_from_exception,
catch_extra_error_codes=catch_extra_error_codes,
sleep_time_generator=sleep_time_generator,
)
@classmethod
def backoff(cls, tries=10, delay=3, backoff=1.1, catch_extra_error_codes=None):
"""
Wrap a callable with retry behavior.
Developers should use CloudRetry.exponential_backoff instead.
This method has been deprecated and will be removed in release 6.0.0, consider using exponential_backoff method instead.
Args:
retries (int): Number of times to retry a failed request before giving up
default=10
delay (int or float): Initial delay between retries in seconds
default=3
backoff (int or float): backoff multiplier e.g. value of 2 will double the delay each retry
default=1.1
catch_extra_error_codes: Additional error messages to catch, in addition to those which may be defined by a subclass of CloudRetry
default=None
Returns:
Callable: A generator that calls the decorated function using an exponential backoff.
"""
# This won't emit a warning (we don't have the context available to us), but will trigger
# sanity failures as we prepare for 6.0.0
ansible_warnings.deprecate(
'CloudRetry.backoff has been deprecated, please use CloudRetry.exponential_backoff instead',
version='6.0.0', collection_name='amazon.aws')
return cls.exponential_backoff(
retries=tries,
delay=delay,
backoff=backoff,
max_delay=None,
catch_extra_error_codes=catch_extra_error_codes,
)

View File

@@ -0,0 +1,229 @@
# -*- coding: utf-8 -*-
#
# Copyright (c) 2017 Willem van Ketwich
#
# This module is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# This software is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this software. If not, see <http://www.gnu.org/licenses/>.
#
# Author:
# - Willem van Ketwich <willem@vanketwich.com.au>
#
# Common functionality to be used by the modules:
# - cloudfront_distribution
# - cloudfront_invalidation
# - cloudfront_origin_access_identity
"""
Common cloudfront facts shared between modules
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
try:
import botocore
except ImportError:
pass
from .ec2 import AWSRetry
from .ec2 import boto3_tag_list_to_ansible_dict
class CloudFrontFactsServiceManager(object):
"""Handles CloudFront Facts Services"""
def __init__(self, module):
self.module = module
self.client = module.client('cloudfront', retry_decorator=AWSRetry.jittered_backoff())
def get_distribution(self, distribution_id):
try:
return self.client.get_distribution(Id=distribution_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing distribution")
def get_distribution_config(self, distribution_id):
try:
return self.client.get_distribution_config(Id=distribution_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing distribution configuration")
def get_origin_access_identity(self, origin_access_identity_id):
try:
return self.client.get_cloud_front_origin_access_identity(Id=origin_access_identity_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing origin access identity")
def get_origin_access_identity_config(self, origin_access_identity_id):
try:
return self.client.get_cloud_front_origin_access_identity_config(Id=origin_access_identity_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing origin access identity configuration")
def get_invalidation(self, distribution_id, invalidation_id):
try:
return self.client.get_invalidation(DistributionId=distribution_id, Id=invalidation_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing invalidation")
def get_streaming_distribution(self, distribution_id):
try:
return self.client.get_streaming_distribution(Id=distribution_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing streaming distribution")
def get_streaming_distribution_config(self, distribution_id):
try:
return self.client.get_streaming_distribution_config(Id=distribution_id, aws_retry=True)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error describing streaming distribution")
def list_origin_access_identities(self):
try:
paginator = self.client.get_paginator('list_cloud_front_origin_access_identities')
result = paginator.paginate().build_full_result().get('CloudFrontOriginAccessIdentityList', {})
return result.get('Items', [])
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error listing cloud front origin access identities")
def list_distributions(self, keyed=True):
try:
paginator = self.client.get_paginator('list_distributions')
result = paginator.paginate().build_full_result().get('DistributionList', {})
distribution_list = result.get('Items', [])
if not keyed:
return distribution_list
return self.keyed_list_helper(distribution_list)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error listing distributions")
def list_distributions_by_web_acl_id(self, web_acl_id):
try:
result = self.client.list_distributions_by_web_acl_id(WebAclId=web_acl_id, aws_retry=True)
distribution_list = result.get('DistributionList', {}).get('Items', [])
return self.keyed_list_helper(distribution_list)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error listing distributions by web acl id")
def list_invalidations(self, distribution_id):
try:
paginator = self.client.get_paginator('list_invalidations')
result = paginator.paginate(DistributionId=distribution_id).build_full_result()
return result.get('InvalidationList', {}).get('Items', [])
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error listing invalidations")
def list_streaming_distributions(self, keyed=True):
try:
paginator = self.client.get_paginator('list_streaming_distributions')
result = paginator.paginate().build_full_result()
streaming_distribution_list = result.get('StreamingDistributionList', {}).get('Items', [])
if not keyed:
return streaming_distribution_list
return self.keyed_list_helper(streaming_distribution_list)
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error listing streaming distributions")
def summary(self):
summary_dict = {}
summary_dict.update(self.summary_get_distribution_list(False))
summary_dict.update(self.summary_get_distribution_list(True))
summary_dict.update(self.summary_get_origin_access_identity_list())
return summary_dict
def summary_get_origin_access_identity_list(self):
try:
origin_access_identity_list = {'origin_access_identities': []}
origin_access_identities = self.list_origin_access_identities()
for origin_access_identity in origin_access_identities:
oai_id = origin_access_identity['Id']
oai_full_response = self.get_origin_access_identity(oai_id)
oai_summary = {'Id': oai_id, 'ETag': oai_full_response['ETag']}
origin_access_identity_list['origin_access_identities'].append(oai_summary)
return origin_access_identity_list
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error generating summary of origin access identities")
def summary_get_distribution_list(self, streaming=False):
try:
list_name = 'streaming_distributions' if streaming else 'distributions'
key_list = ['Id', 'ARN', 'Status', 'LastModifiedTime', 'DomainName', 'Comment', 'PriceClass', 'Enabled']
distribution_list = {list_name: []}
distributions = self.list_streaming_distributions(False) if streaming else self.list_distributions(False)
for dist in distributions:
temp_distribution = {}
for key_name in key_list:
temp_distribution[key_name] = dist[key_name]
temp_distribution['Aliases'] = list(dist['Aliases'].get('Items', []))
temp_distribution['ETag'] = self.get_etag_from_distribution_id(dist['Id'], streaming)
if not streaming:
temp_distribution['WebACLId'] = dist['WebACLId']
invalidation_ids = self.get_list_of_invalidation_ids_from_distribution_id(dist['Id'])
if invalidation_ids:
temp_distribution['Invalidations'] = invalidation_ids
resource_tags = self.client.list_tags_for_resource(Resource=dist['ARN'], aws_retry=True)
temp_distribution['Tags'] = boto3_tag_list_to_ansible_dict(resource_tags['Tags'].get('Items', []))
distribution_list[list_name].append(temp_distribution)
return distribution_list
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
self.module.fail_json_aws(e, msg="Error generating summary of distributions")
def get_etag_from_distribution_id(self, distribution_id, streaming):
distribution = {}
if not streaming:
distribution = self.get_distribution(distribution_id)
else:
distribution = self.get_streaming_distribution(distribution_id)
return distribution['ETag']
def get_list_of_invalidation_ids_from_distribution_id(self, distribution_id):
try:
invalidation_ids = []
invalidations = self.list_invalidations(distribution_id)
for invalidation in invalidations:
invalidation_ids.append(invalidation['Id'])
return invalidation_ids
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error getting list of invalidation ids")
def get_distribution_id_from_domain_name(self, domain_name):
try:
distribution_id = ""
distributions = self.list_distributions(False)
distributions += self.list_streaming_distributions(False)
for dist in distributions:
if 'Items' in dist['Aliases']:
for alias in dist['Aliases']['Items']:
if str(alias).lower() == domain_name.lower():
distribution_id = dist['Id']
break
return distribution_id
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error getting distribution id from domain name")
def get_aliases_from_distribution_id(self, distribution_id):
try:
distribution = self.get_distribution(distribution_id)
return distribution['DistributionConfig']['Aliases'].get('Items', [])
except botocore.exceptions.ClientError as e:
self.module.fail_json_aws(e, msg="Error getting list of aliases from distribution_id")
def keyed_list_helper(self, list_to_key):
keyed_list = dict()
for item in list_to_key:
distribution_id = item['Id']
if 'Items' in item['Aliases']:
aliases = item['Aliases']['Items']
for alias in aliases:
keyed_list.update({alias: item})
keyed_list.update({distribution_id: item})
return keyed_list

View File

@@ -0,0 +1,77 @@
#
# Copyright 2017 Michael De La Rue | Ansible
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
"""This module adds shared support for generic Amazon AWS modules
In order to use this module, include it as part of a custom
module as shown below.
from ansible_collections.amazon.aws.plugins.module_utils.core import AnsibleAWSModule
module = AnsibleAWSModule(argument_spec=dictionary, supports_check_mode=boolean
mutually_exclusive=list1, required_together=list2)
The 'AnsibleAWSModule' module provides similar, but more restricted,
interfaces to the normal Ansible module. It also includes the
additional methods for connecting to AWS using the standard module arguments
m.resource('lambda') # - get an AWS connection as a boto3 resource.
or
m.client('sts') # - get an AWS connection as a boto3 client.
To make use of AWSRetry easier, it can now be wrapped around any call from a
module-created client. To add retries to a client, create a client:
m.client('ec2', retry_decorator=AWSRetry.jittered_backoff(retries=10))
Any calls from that client can be made to use the decorator passed at call-time
using the `aws_retry` argument. By default, no retries are used.
ec2 = m.client('ec2', retry_decorator=AWSRetry.jittered_backoff(retries=10))
ec2.describe_instances(InstanceIds=['i-123456789'], aws_retry=True)
The call will be retried the specified number of times, so the calling functions
don't need to be wrapped in the backoff decorator.
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.arn
from .arn import parse_aws_arn # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.botocore
from .botocore import HAS_BOTO3 # pylint: disable=unused-import
from .botocore import is_boto3_error_code # pylint: disable=unused-import
from .botocore import is_boto3_error_message # pylint: disable=unused-import
from .botocore import get_boto3_client_method_parameters # pylint: disable=unused-import
from .botocore import normalize_boto3_result # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.modules
from .modules import AnsibleAWSModule # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.modules
from .transformation import scrub_none_parameters # pylint: disable=unused-import
# We will also export HAS_BOTO3 so end user modules can use it.
__all__ = ('AnsibleAWSModule', 'HAS_BOTO3', 'is_boto3_error_code', 'is_boto3_error_message')
class AnsibleAWSError(Exception):
pass

View File

@@ -0,0 +1,89 @@
# Copyright (c) 2017 Ansible Project
#
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
"""
This module adds shared support for Direct Connect modules.
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import traceback
try:
import botocore
except ImportError:
pass
from .ec2 import AWSRetry
class DirectConnectError(Exception):
def __init__(self, msg, last_traceback=None, exception=None):
self.msg = msg
self.last_traceback = last_traceback
self.exception = exception
def delete_connection(client, connection_id):
try:
AWSRetry.jittered_backoff()(client.delete_connection)(connectionId=connection_id)
except botocore.exceptions.ClientError as e:
raise DirectConnectError(msg="Failed to delete DirectConnection {0}.".format(connection_id),
last_traceback=traceback.format_exc(),
exception=e)
def associate_connection_and_lag(client, connection_id, lag_id):
try:
AWSRetry.jittered_backoff()(client.associate_connection_with_lag)(connectionId=connection_id,
lagId=lag_id)
except botocore.exceptions.ClientError as e:
raise DirectConnectError(msg="Failed to associate Direct Connect connection {0}"
" with link aggregation group {1}.".format(connection_id, lag_id),
last_traceback=traceback.format_exc(),
exception=e)
def disassociate_connection_and_lag(client, connection_id, lag_id):
try:
AWSRetry.jittered_backoff()(client.disassociate_connection_from_lag)(connectionId=connection_id,
lagId=lag_id)
except botocore.exceptions.ClientError as e:
raise DirectConnectError(msg="Failed to disassociate Direct Connect connection {0}"
" from link aggregation group {1}.".format(connection_id, lag_id),
last_traceback=traceback.format_exc(),
exception=e)
def delete_virtual_interface(client, virtual_interface):
try:
AWSRetry.jittered_backoff()(client.delete_virtual_interface)(virtualInterfaceId=virtual_interface)
except botocore.exceptions.ClientError as e:
raise DirectConnectError(msg="Could not delete virtual interface {0}".format(virtual_interface),
last_traceback=traceback.format_exc(),
exception=e)

View File

@@ -0,0 +1,310 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Michael DeHaan <michael.dehaan@gmail.com>, 2012-2013
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
This module adds helper functions for various EC2 specific services.
It also includes a large number of imports for functions which historically
lived here. Most of these functions were not specific to EC2, they ended
up in this module because "that's where the AWS code was" (originally).
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import re
from ansible.module_utils.ansible_release import __version__
from ansible.module_utils.six import string_types
from ansible.module_utils.six import integer_types
# Used to live here, moved into ansible.module_utils.common.dict_transformations
from ansible.module_utils.common.dict_transformations import _camel_to_snake # pylint: disable=unused-import
from ansible.module_utils.common.dict_transformations import _snake_to_camel # pylint: disable=unused-import
from ansible.module_utils.common.dict_transformations import camel_dict_to_snake_dict # pylint: disable=unused-import
from ansible.module_utils.common.dict_transformations import snake_dict_to_camel_dict # pylint: disable=unused-import
# Used to live here, moved into # ansible_collections.amazon.aws.plugins.module_utils.arn
from .arn import is_outpost_arn as is_outposts_arn # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.botocore
from .botocore import HAS_BOTO3 # pylint: disable=unused-import
from .botocore import boto3_conn # pylint: disable=unused-import
from .botocore import boto3_inventory_conn # pylint: disable=unused-import
from .botocore import boto_exception # pylint: disable=unused-import
from .botocore import get_aws_region # pylint: disable=unused-import
from .botocore import get_aws_connection_info # pylint: disable=unused-import
from .botocore import paginated_query_with_retries
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.botocore
from .core import AnsibleAWSError # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.modules
# The names have been changed in .modules to better reflect their applicability.
from .modules import _aws_common_argument_spec as aws_common_argument_spec # pylint: disable=unused-import
from .modules import aws_argument_spec as ec2_argument_spec # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.tagging
from .tagging import ansible_dict_to_boto3_tag_list # pylint: disable=unused-import
from .tagging import boto3_tag_list_to_ansible_dict # pylint: disable=unused-import
from .tagging import compare_aws_tags # pylint: disable=unused-import
# Used to live here, moved into ansible_collections.amazon.aws.plugins.module_utils.transformation
from .transformation import ansible_dict_to_boto3_filter_list # pylint: disable=unused-import
from .transformation import map_complex_type # pylint: disable=unused-import
# Used to live here, moved into # ansible_collections.amazon.aws.plugins.module_utils.policy
from .policy import _py3cmp as py3cmp # pylint: disable=unused-import
from .policy import compare_policies # pylint: disable=unused-import
from .policy import sort_json_policy_dict # pylint: disable=unused-import
# Used to live here, moved into # ansible_collections.amazon.aws.plugins.module_utils.retries
from .retries import AWSRetry # pylint: disable=unused-import
try:
import botocore
except ImportError:
pass # Handled by HAS_BOTO3
def get_ec2_security_group_ids_from_names(sec_group_list, ec2_connection, vpc_id=None, boto3=None):
""" Return list of security group IDs from security group names. Note that security group names are not unique
across VPCs. If a name exists across multiple VPCs and no VPC ID is supplied, all matching IDs will be returned. This
will probably lead to a boto exception if you attempt to assign both IDs to a resource so ensure you wrap the call in
a try block
"""
def get_sg_name(sg, boto3=None):
return str(sg['GroupName'])
def get_sg_id(sg, boto3=None):
return str(sg['GroupId'])
sec_group_id_list = []
if isinstance(sec_group_list, string_types):
sec_group_list = [sec_group_list]
# Get all security groups
if vpc_id:
filters = [
{
'Name': 'vpc-id',
'Values': [
vpc_id,
]
}
]
all_sec_groups = ec2_connection.describe_security_groups(Filters=filters)['SecurityGroups']
else:
all_sec_groups = ec2_connection.describe_security_groups()['SecurityGroups']
unmatched = set(sec_group_list).difference(str(get_sg_name(all_sg, boto3)) for all_sg in all_sec_groups)
sec_group_name_list = list(set(sec_group_list) - set(unmatched))
if len(unmatched) > 0:
# If we have unmatched names that look like an ID, assume they are
sec_group_id_list[:] = [sg for sg in unmatched if re.match('sg-[a-fA-F0-9]+$', sg)]
still_unmatched = [sg for sg in unmatched if not re.match('sg-[a-fA-F0-9]+$', sg)]
if len(still_unmatched) > 0:
raise ValueError("The following group names are not valid: %s" % ', '.join(still_unmatched))
sec_group_id_list += [get_sg_id(all_sg) for all_sg in all_sec_groups if get_sg_name(all_sg) in sec_group_name_list]
return sec_group_id_list
def add_ec2_tags(client, module, resource_id, tags_to_set, retry_codes=None):
"""
Sets Tags on an EC2 resource.
:param client: an EC2 boto3 client
:param module: an AnsibleAWSModule object
:param resource_id: the identifier for the resource
:param tags_to_set: A dictionary of key/value pairs to set
:param retry_codes: additional boto3 error codes to trigger retries
"""
if not tags_to_set:
return False
if module.check_mode:
return True
if not retry_codes:
retry_codes = []
try:
tags_to_add = ansible_dict_to_boto3_tag_list(tags_to_set)
AWSRetry.jittered_backoff(retries=10, catch_extra_error_codes=retry_codes)(
client.create_tags
)(
Resources=[resource_id], Tags=tags_to_add
)
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:
module.fail_json_aws(e, msg="Unable to add tags {0} to {1}".format(tags_to_set, resource_id))
return True
def remove_ec2_tags(client, module, resource_id, tags_to_unset, retry_codes=None):
"""
Removes Tags from an EC2 resource.
:param client: an EC2 boto3 client
:param module: an AnsibleAWSModule object
:param resource_id: the identifier for the resource
:param tags_to_unset: a list of tag keys to removes
:param retry_codes: additional boto3 error codes to trigger retries
"""
if not tags_to_unset:
return False
if module.check_mode:
return True
if not retry_codes:
retry_codes = []
tags_to_remove = [dict(Key=tagkey) for tagkey in tags_to_unset]
try:
AWSRetry.jittered_backoff(retries=10, catch_extra_error_codes=retry_codes)(
client.delete_tags
)(
Resources=[resource_id], Tags=tags_to_remove
)
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:
module.fail_json_aws(e, msg="Unable to delete tags {0} from {1}".format(tags_to_unset, resource_id))
return True
def describe_ec2_tags(client, module, resource_id, resource_type=None, retry_codes=None):
"""
Performs a paginated search of EC2 resource tags.
:param client: an EC2 boto3 client
:param module: an AnsibleAWSModule object
:param resource_id: the identifier for the resource
:param resource_type: the type of the resource
:param retry_codes: additional boto3 error codes to trigger retries
"""
filters = {'resource-id': resource_id}
if resource_type:
filters['resource-type'] = resource_type
filters = ansible_dict_to_boto3_filter_list(filters)
if not retry_codes:
retry_codes = []
try:
retry_decorator = AWSRetry.jittered_backoff(retries=10, catch_extra_error_codes=retry_codes)
results = paginated_query_with_retries(client, 'describe_tags', retry_decorator=retry_decorator,
Filters=filters)
return boto3_tag_list_to_ansible_dict(results.get('Tags', None))
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:
module.fail_json_aws(e, msg="Failed to describe tags for EC2 Resource: {0}".format(resource_id))
def ensure_ec2_tags(client, module, resource_id, resource_type=None, tags=None, purge_tags=True, retry_codes=None):
"""
Updates the tags on an EC2 resource.
To remove all tags the tags parameter must be explicitly set to an empty dictionary.
:param client: an EC2 boto3 client
:param module: an AnsibleAWSModule object
:param resource_id: the identifier for the resource
:param resource_type: the type of the resource
:param tags: the Tags to apply to the resource
:param purge_tags: whether tags missing from the tag list should be removed
:param retry_codes: additional boto3 error codes to trigger retries
:return: changed: returns True if the tags are changed
"""
if tags is None:
return False
if not retry_codes:
retry_codes = []
changed = False
current_tags = describe_ec2_tags(client, module, resource_id, resource_type, retry_codes)
tags_to_set, tags_to_unset = compare_aws_tags(current_tags, tags, purge_tags)
if purge_tags and not tags:
tags_to_unset = current_tags
changed |= remove_ec2_tags(client, module, resource_id, tags_to_unset, retry_codes)
changed |= add_ec2_tags(client, module, resource_id, tags_to_set, retry_codes)
return changed
def normalize_ec2_vpc_dhcp_config(option_config):
"""
The boto2 module returned a config dict, but boto3 returns a list of dicts
Make the data we return look like the old way, so we don't break users.
This is also much more user-friendly.
boto3:
'DhcpConfigurations': [
{'Key': 'domain-name', 'Values': [{'Value': 'us-west-2.compute.internal'}]},
{'Key': 'domain-name-servers', 'Values': [{'Value': 'AmazonProvidedDNS'}]},
{'Key': 'netbios-name-servers', 'Values': [{'Value': '1.2.3.4'}, {'Value': '5.6.7.8'}]},
{'Key': 'netbios-node-type', 'Values': [1]},
{'Key': 'ntp-servers', 'Values': [{'Value': '1.2.3.4'}, {'Value': '5.6.7.8'}]}
],
The module historically returned:
"new_options": {
"domain-name": "ec2.internal",
"domain-name-servers": ["AmazonProvidedDNS"],
"netbios-name-servers": ["10.0.0.1", "10.0.1.1"],
"netbios-node-type": "1",
"ntp-servers": ["10.0.0.2", "10.0.1.2"]
},
"""
config_data = {}
if len(option_config) == 0:
# If there is no provided config, return the empty dictionary
return config_data
for config_item in option_config:
# Handle single value keys
if config_item['Key'] == 'netbios-node-type':
if isinstance(config_item['Values'], integer_types):
config_data['netbios-node-type'] = str((config_item['Values']))
elif isinstance(config_item['Values'], list):
config_data['netbios-node-type'] = str((config_item['Values'][0]['Value']))
# Handle actual lists of values
for option in ['domain-name', 'domain-name-servers', 'ntp-servers', 'netbios-name-servers']:
if config_item['Key'] == option:
config_data[option] = [val['Value'] for val in config_item['Values']]
return config_data

View File

@@ -0,0 +1,109 @@
# Copyright (c) 2017 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
try:
from botocore.exceptions import BotoCoreError, ClientError
except ImportError:
pass
from .core import is_boto3_error_code
from .ec2 import AWSRetry
def get_elb(connection, module, elb_name):
"""
Get an ELB based on name. If not found, return None.
:param connection: AWS boto3 elbv2 connection
:param module: Ansible module
:param elb_name: Name of load balancer to get
:return: boto3 ELB dict or None if not found
"""
try:
return _get_elb(connection, module, elb_name)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e)
@AWSRetry.jittered_backoff()
def _get_elb(connection, module, elb_name):
"""
Get an ELB based on name using AWSRetry. If not found, return None.
:param connection: AWS boto3 elbv2 connection
:param module: Ansible module
:param elb_name: Name of load balancer to get
:return: boto3 ELB dict or None if not found
"""
try:
load_balancer_paginator = connection.get_paginator('describe_load_balancers')
return (load_balancer_paginator.paginate(Names=[elb_name]).build_full_result())['LoadBalancers'][0]
except is_boto3_error_code('LoadBalancerNotFound'):
return None
def get_elb_listener(connection, module, elb_arn, listener_port):
"""
Get an ELB listener based on the port provided. If not found, return None.
:param connection: AWS boto3 elbv2 connection
:param module: Ansible module
:param elb_arn: ARN of the ELB to look at
:param listener_port: Port of the listener to look for
:return: boto3 ELB listener dict or None if not found
"""
try:
listener_paginator = connection.get_paginator('describe_listeners')
listeners = (AWSRetry.jittered_backoff()(listener_paginator.paginate)(LoadBalancerArn=elb_arn).build_full_result())['Listeners']
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e)
l = None
for listener in listeners:
if listener['Port'] == listener_port:
l = listener
break
return l
def get_elb_listener_rules(connection, module, listener_arn):
"""
Get rules for a particular ELB listener using the listener ARN.
:param connection: AWS boto3 elbv2 connection
:param module: Ansible module
:param listener_arn: ARN of the ELB listener
:return: boto3 ELB rules list
"""
try:
return AWSRetry.jittered_backoff()(connection.describe_rules)(ListenerArn=listener_arn)['Rules']
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e)
def convert_tg_name_to_arn(connection, module, tg_name):
"""
Get ARN of a target group using the target group's name
:param connection: AWS boto3 elbv2 connection
:param module: Ansible module
:param tg_name: Name of the target group
:return: target group ARN string
"""
try:
response = AWSRetry.jittered_backoff()(connection.describe_target_groups)(Names=[tg_name])
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e)
tg_arn = response['TargetGroups'][0]['TargetGroupArn']
return tg_arn

View File

@@ -0,0 +1,75 @@
# Copyright (c) 2017 Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
try:
import botocore
except ImportError:
pass
from ansible.module_utils._text import to_native
from .ec2 import AWSRetry
from .core import is_boto3_error_code
from .core import parse_aws_arn
def get_aws_account_id(module):
""" Given an AnsibleAWSModule instance, get the active AWS account ID
"""
return get_aws_account_info(module)[0]
def get_aws_account_info(module):
"""Given an AnsibleAWSModule instance, return the account information
(account id and partition) we are currently working on
get_account_info tries too find out the account that we are working
on. It's not guaranteed that this will be easy so we try in
several different ways. Giving either IAM or STS privileges to
the account should be enough to permit this.
Tries:
- sts:GetCallerIdentity
- iam:GetUser
- sts:DecodeAuthorizationMessage
"""
account_id = None
partition = None
try:
sts_client = module.client('sts', retry_decorator=AWSRetry.jittered_backoff())
caller_id = sts_client.get_caller_identity(aws_retry=True)
account_id = caller_id.get('Account')
partition = caller_id.get('Arn').split(':')[1]
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError):
try:
iam_client = module.client('iam', retry_decorator=AWSRetry.jittered_backoff())
_arn, partition, _service, _reg, account_id, _resource = iam_client.get_user(aws_retry=True)['User']['Arn'].split(':')
except is_boto3_error_code('AccessDenied') as e:
try:
except_msg = to_native(e.message)
except AttributeError:
except_msg = to_native(e)
result = parse_aws_arn(except_msg)
if result is None or result['service'] != 'iam':
module.fail_json_aws(
e,
msg="Failed to get AWS account information, Try allowing sts:GetCallerIdentity or iam:GetUser permissions."
)
account_id = result.get('account_id')
partition = result.get('partition')
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e: # pylint: disable=duplicate-except
module.fail_json_aws(
e,
msg="Failed to get AWS account information, Try allowing sts:GetCallerIdentity or iam:GetUser permissions."
)
if account_id is None or partition is None:
module.fail_json(
msg="Failed to get AWS account information, Try allowing sts:GetCallerIdentity or iam:GetUser permissions."
)
return (to_native(account_id), to_native(partition))

View File

@@ -0,0 +1,451 @@
#
# Copyright 2017 Michael De La Rue | Ansible
#
# This file is part of Ansible
#
# Ansible is free software: you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation, either version 3 of the License, or
# (at your option) any later version.
#
# Ansible is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with Ansible. If not, see <http://www.gnu.org/licenses/>.
"""This module adds shared support for generic Amazon AWS modules
In order to use this module, include it as part of a custom
module as shown below.
from ansible_collections.amazon.aws.plugins.module_utils.modules import AnsibleAWSModule
module = AnsibleAWSModule(argument_spec=dictionary, supports_check_mode=boolean
mutually_exclusive=list1, required_together=list2)
The 'AnsibleAWSModule' module provides similar, but more restricted,
interfaces to the normal Ansible module. It also includes the
additional methods for connecting to AWS using the standard module arguments
m.resource('lambda') # - get an AWS connection as a boto3 resource.
or
m.client('sts') # - get an AWS connection as a boto3 client.
To make use of AWSRetry easier, it can now be wrapped around any call from a
module-created client. To add retries to a client, create a client:
m.client('ec2', retry_decorator=AWSRetry.jittered_backoff(retries=10))
Any calls from that client can be made to use the decorator passed at call-time
using the `aws_retry` argument. By default, no retries are used.
ec2 = m.client('ec2', retry_decorator=AWSRetry.jittered_backoff(retries=10))
ec2.describe_instances(InstanceIds=['i-123456789'], aws_retry=True)
The call will be retried the specified number of times, so the calling functions
don't need to be wrapped in the backoff decorator.
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from functools import wraps
import logging
import os
import re
import traceback
try:
from cStringIO import StringIO
except ImportError:
# Python 3
from io import StringIO
from ansible.module_utils.basic import AnsibleModule
from ansible.module_utils.basic import env_fallback
from ansible.module_utils.basic import missing_required_lib
from ansible.module_utils.common.dict_transformations import camel_dict_to_snake_dict
from ansible.module_utils._text import to_native
from .botocore import HAS_BOTO3
from .botocore import boto3_conn
from .botocore import get_aws_connection_info
from .botocore import get_aws_region
from .botocore import gather_sdk_versions
from .version import LooseVersion
# Currently only AnsibleAWSModule. However we have a lot of Copy and Paste code
# for Inventory and Lookup modules which we should refactor
class AnsibleAWSModule(object):
"""An ansible module class for AWS modules
AnsibleAWSModule provides an a class for building modules which
connect to Amazon Web Services. The interface is currently more
restricted than the basic module class with the aim that later the
basic module class can be reduced. If you find that any key
feature is missing please contact the author/Ansible AWS team
(available on #ansible-aws on IRC) to request the additional
features needed.
"""
default_settings = {
"default_args": True,
"check_boto3": True,
"auto_retry": True,
"module_class": AnsibleModule
}
def __init__(self, **kwargs):
local_settings = {}
for key in AnsibleAWSModule.default_settings:
try:
local_settings[key] = kwargs.pop(key)
except KeyError:
local_settings[key] = AnsibleAWSModule.default_settings[key]
self.settings = local_settings
if local_settings["default_args"]:
argument_spec_full = aws_argument_spec()
try:
argument_spec_full.update(kwargs["argument_spec"])
except (TypeError, NameError):
pass
kwargs["argument_spec"] = argument_spec_full
self._module = AnsibleAWSModule.default_settings["module_class"](**kwargs)
if local_settings["check_boto3"]:
if not HAS_BOTO3:
self._module.fail_json(
msg=missing_required_lib('botocore and boto3'))
if not self.botocore_at_least('1.21.0'):
self.warn('botocore < 1.21.0 is not supported or tested.'
' Some features may not work.')
if not self.boto3_at_least("1.18.0"):
self.warn('boto3 < 1.18.0 is not supported or tested.'
' Some features may not work.')
deprecated_vars = {'EC2_REGION', 'EC2_SECURITY_TOKEN', 'EC2_SECRET_KEY', 'EC2_ACCESS_KEY',
'EC2_URL', 'S3_URL'}
if deprecated_vars.intersection(set(os.environ.keys())):
self._module.deprecate(
"Support for the 'EC2_REGION', 'EC2_ACCESS_KEY', 'EC2_SECRET_KEY', "
"'EC2_SECURITY_TOKEN', 'EC2_URL', and 'S3_URL' environment "
"variables has been deprecated. "
"These variables are currently used for all AWS services which can "
"cause confusion. We recomend using the relevant module "
"parameters or alternatively the 'AWS_REGION', 'AWS_ACCESS_KEY_ID', "
"'AWS_SECRET_ACCESS_KEY', 'AWS_SESSION_TOKEN', and 'AWS_URL' "
"environment variables can be used instead.",
date='2024-12-01', collection_name='amazon.aws',
)
if 'AWS_SECURITY_TOKEN' in os.environ.keys():
self._module.deprecate(
"Support for the 'AWS_SECURITY_TOKEN' environment variable "
"has been deprecated. This variable was based on the original "
"boto SDK, support for which has now been dropped. "
"We recommend using the 'session_token' module parameter "
"or alternatively the 'AWS_SESSION_TOKEN' environment variable "
"can be used instead.",
date='2024-12-01', collection_name='amazon.aws',
)
self.check_mode = self._module.check_mode
self._diff = self._module._diff
self._name = self._module._name
self._botocore_endpoint_log_stream = StringIO()
self.logger = None
if self.params.get('debug_botocore_endpoint_logs'):
self.logger = logging.getLogger('botocore.endpoint')
self.logger.setLevel(logging.DEBUG)
self.logger.addHandler(logging.StreamHandler(self._botocore_endpoint_log_stream))
@property
def params(self):
return self._module.params
def _get_resource_action_list(self):
actions = []
for ln in self._botocore_endpoint_log_stream.getvalue().split('\n'):
ln = ln.strip()
if not ln:
continue
found_operational_request = re.search(r"OperationModel\(name=.*?\)", ln)
if found_operational_request:
operation_request = found_operational_request.group(0)[20:-1]
resource = re.search(r"https://.*?\.", ln).group(0)[8:-1]
actions.append("{0}:{1}".format(resource, operation_request))
return list(set(actions))
def exit_json(self, *args, **kwargs):
if self.params.get('debug_botocore_endpoint_logs'):
kwargs['resource_actions'] = self._get_resource_action_list()
return self._module.exit_json(*args, **kwargs)
def fail_json(self, *args, **kwargs):
if self.params.get('debug_botocore_endpoint_logs'):
kwargs['resource_actions'] = self._get_resource_action_list()
return self._module.fail_json(*args, **kwargs)
def debug(self, *args, **kwargs):
return self._module.debug(*args, **kwargs)
def warn(self, *args, **kwargs):
return self._module.warn(*args, **kwargs)
def deprecate(self, *args, **kwargs):
return self._module.deprecate(*args, **kwargs)
def boolean(self, *args, **kwargs):
return self._module.boolean(*args, **kwargs)
def md5(self, *args, **kwargs):
return self._module.md5(*args, **kwargs)
def client(self, service, retry_decorator=None):
region, endpoint_url, aws_connect_kwargs = get_aws_connection_info(self, boto3=True)
conn = boto3_conn(self, conn_type='client', resource=service,
region=region, endpoint=endpoint_url, **aws_connect_kwargs)
return conn if retry_decorator is None else _RetryingBotoClientWrapper(conn, retry_decorator)
def resource(self, service):
region, endpoint_url, aws_connect_kwargs = get_aws_connection_info(self, boto3=True)
return boto3_conn(self, conn_type='resource', resource=service,
region=region, endpoint=endpoint_url, **aws_connect_kwargs)
@property
def region(self):
return get_aws_region(self, True)
def fail_json_aws(self, exception, msg=None, **kwargs):
"""call fail_json with processed exception
function for converting exceptions thrown by AWS SDK modules,
botocore, boto3 and boto, into nice error messages.
"""
last_traceback = traceback.format_exc()
# to_native is trusted to handle exceptions that str() could
# convert to text.
try:
except_msg = to_native(exception.message)
except AttributeError:
except_msg = to_native(exception)
if msg is not None:
message = '{0}: {1}'.format(msg, except_msg)
else:
message = except_msg
try:
response = exception.response
except AttributeError:
response = None
failure = dict(
msg=message,
exception=last_traceback,
**self._gather_versions()
)
failure.update(kwargs)
if response is not None:
failure.update(**camel_dict_to_snake_dict(response))
self.fail_json(**failure)
def _gather_versions(self):
"""Gather AWS SDK (boto3 and botocore) dependency versions
Returns {'boto3_version': str, 'botocore_version': str}
Returns {} if either is not installed
"""
return gather_sdk_versions()
def require_boto3_at_least(self, desired, **kwargs):
"""Check if the available boto3 version is greater than or equal to a desired version.
calls fail_json() when the boto3 version is less than the desired
version
Usage:
module.require_boto3_at_least("1.2.3", reason="to update tags")
module.require_boto3_at_least("1.1.1")
:param desired the minimum desired version
:param reason why the version is required (optional)
"""
if not self.boto3_at_least(desired):
self._module.fail_json(
msg=missing_required_lib('boto3>={0}'.format(desired), **kwargs),
**self._gather_versions()
)
def boto3_at_least(self, desired):
"""Check if the available boto3 version is greater than or equal to a desired version.
Usage:
if module.params.get('assign_ipv6_address') and not module.boto3_at_least('1.4.4'):
# conditionally fail on old boto3 versions if a specific feature is not supported
module.fail_json(msg="Boto3 can't deal with EC2 IPv6 addresses before version 1.4.4.")
"""
existing = self._gather_versions()
return LooseVersion(existing['boto3_version']) >= LooseVersion(desired)
def require_botocore_at_least(self, desired, **kwargs):
"""Check if the available botocore version is greater than or equal to a desired version.
calls fail_json() when the botocore version is less than the desired
version
Usage:
module.require_botocore_at_least("1.2.3", reason="to update tags")
module.require_botocore_at_least("1.1.1")
:param desired the minimum desired version
:param reason why the version is required (optional)
"""
if not self.botocore_at_least(desired):
self._module.fail_json(
msg=missing_required_lib('botocore>={0}'.format(desired), **kwargs),
**self._gather_versions()
)
def botocore_at_least(self, desired):
"""Check if the available botocore version is greater than or equal to a desired version.
Usage:
if not module.botocore_at_least('1.2.3'):
module.fail_json(msg='The Serverless Elastic Load Compute Service is not in botocore before v1.2.3')
if not module.botocore_at_least('1.5.3'):
module.warn('Botocore did not include waiters for Service X before 1.5.3. '
'To wait until Service X resources are fully available, update botocore.')
"""
existing = self._gather_versions()
return LooseVersion(existing['botocore_version']) >= LooseVersion(desired)
class _RetryingBotoClientWrapper(object):
__never_wait = (
'get_paginator', 'can_paginate',
'get_waiter', 'generate_presigned_url',
)
def __init__(self, client, retry):
self.client = client
self.retry = retry
def _create_optional_retry_wrapper_function(self, unwrapped):
retrying_wrapper = self.retry(unwrapped)
@wraps(unwrapped)
def deciding_wrapper(aws_retry=False, *args, **kwargs):
if aws_retry:
return retrying_wrapper(*args, **kwargs)
else:
return unwrapped(*args, **kwargs)
return deciding_wrapper
def __getattr__(self, name):
unwrapped = getattr(self.client, name)
if name in self.__never_wait:
return unwrapped
elif callable(unwrapped):
wrapped = self._create_optional_retry_wrapper_function(unwrapped)
setattr(self, name, wrapped)
return wrapped
else:
return unwrapped
def _aws_common_argument_spec():
"""
This does not include 'region' as some AWS APIs don't require a
region. However, it's not recommended to do this as it means module_defaults
can't include the region parameter.
"""
return dict(
access_key=dict(
aliases=['aws_access_key_id', 'aws_access_key', 'ec2_access_key'],
deprecated_aliases=[
dict(name='ec2_access_key', date='2024-12-01', collection_name='amazon.aws'),
],
fallback=(env_fallback, ['AWS_ACCESS_KEY_ID', 'AWS_ACCESS_KEY', 'EC2_ACCESS_KEY']),
no_log=False,
),
secret_key=dict(
aliases=['aws_secret_access_key', 'aws_secret_key', 'ec2_secret_key'],
deprecated_aliases=[
dict(name='ec2_secret_key', date='2024-12-01', collection_name='amazon.aws'),
],
fallback=(env_fallback, ['AWS_SECRET_ACCESS_KEY', 'AWS_SECRET_KEY', 'EC2_SECRET_KEY']),
no_log=True,
),
session_token=dict(
aliases=['aws_session_token', 'security_token', 'access_token', 'aws_security_token'],
deprecated_aliases=[
dict(name='access_token', date='2024-12-01', collection_name='amazon.aws'),
dict(name='security_token', date='2024-12-01', collection_name='amazon.aws'),
dict(name='aws_security_token', date='2024-12-01', collection_name='amazon.aws'),
],
fallback=(env_fallback, ['AWS_SESSION_TOKEN', 'AWS_SECURITY_TOKEN', 'EC2_SECURITY_TOKEN']),
no_log=True,
),
profile=dict(
aliases=['aws_profile'],
fallback=(env_fallback, ['AWS_PROFILE', 'AWS_DEFAULT_PROFILE']),
),
endpoint_url=dict(
aliases=['aws_endpoint_url', 'ec2_url', 's3_url'],
deprecated_aliases=[
dict(name='ec2_url', date='2024-12-01', collection_name='amazon.aws'),
dict(name='s3_url', date='2024-12-01', collection_name='amazon.aws'),
],
fallback=(env_fallback, ['AWS_URL', 'EC2_URL', 'S3_URL']),
),
validate_certs=dict(
type='bool',
default=True,
),
aws_ca_bundle=dict(
type='path',
fallback=(env_fallback, ['AWS_CA_BUNDLE']),
),
aws_config=dict(
type='dict',
),
debug_botocore_endpoint_logs=dict(
type='bool',
default=False,
fallback=(env_fallback, ['ANSIBLE_DEBUG_BOTOCORE_LOGS']),
),
)
def aws_argument_spec():
"""
Returns a dictionary containing the argument_spec common to all AWS modules.
"""
region_spec = dict(
region=dict(
aliases=['aws_region', 'ec2_region'],
deprecated_aliases=[
dict(name='ec2_region', date='2024-12-01', collection_name='amazon.aws'),
],
fallback=(env_fallback, ['AWS_REGION', 'AWS_DEFAULT_REGION', 'EC2_REGION']),
),
)
spec = _aws_common_argument_spec()
spec.update(region_spec)
return spec

View File

@@ -0,0 +1,179 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Michael DeHaan <michael.dehaan@gmail.com>, 2012-2013
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from functools import cmp_to_key
from ansible.module_utils._text import to_text
from ansible.module_utils.six import binary_type
from ansible.module_utils.six import string_types
def _hashable_policy(policy, policy_list):
"""
Takes a policy and returns a list, the contents of which are all hashable and sorted.
Example input policy:
{'Version': '2012-10-17',
'Statement': [{'Action': 's3:PutObjectAcl',
'Sid': 'AddCannedAcl2',
'Resource': 'arn:aws:s3:::test_policy/*',
'Effect': 'Allow',
'Principal': {'AWS': ['arn:aws:iam::XXXXXXXXXXXX:user/username1', 'arn:aws:iam::XXXXXXXXXXXX:user/username2']}
}]}
Returned value:
[('Statement', ((('Action', ('s3:PutObjectAcl',)),
('Effect', ('Allow',)),
('Principal', ('AWS', (('arn:aws:iam::XXXXXXXXXXXX:user/username1',), ('arn:aws:iam::XXXXXXXXXXXX:user/username2',)))),
('Resource', ('arn:aws:s3:::test_policy/*',)), ('Sid', ('AddCannedAcl2',)))),
('Version', ('2012-10-17',)))]
"""
# Amazon will automatically convert bool and int to strings for us
if isinstance(policy, bool):
return tuple([str(policy).lower()])
elif isinstance(policy, int):
return tuple([str(policy)])
if isinstance(policy, list):
for each in policy:
tupleified = _hashable_policy(each, [])
if isinstance(tupleified, list):
tupleified = tuple(tupleified)
policy_list.append(tupleified)
elif isinstance(policy, string_types) or isinstance(policy, binary_type):
policy = to_text(policy)
# convert root account ARNs to just account IDs
if policy.startswith('arn:aws:iam::') and policy.endswith(':root'):
policy = policy.split(':')[4]
return [policy]
elif isinstance(policy, dict):
sorted_keys = list(policy.keys())
sorted_keys.sort()
for key in sorted_keys:
element = policy[key]
# Special case defined in
# https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_principal.html
if key in ["NotPrincipal", "Principal"] and policy[key] == "*":
element = {"AWS": "*"}
tupleified = _hashable_policy(element, [])
if isinstance(tupleified, list):
tupleified = tuple(tupleified)
policy_list.append((key, tupleified))
# ensure we aren't returning deeply nested structures of length 1
if len(policy_list) == 1 and isinstance(policy_list[0], tuple):
policy_list = policy_list[0]
if isinstance(policy_list, list):
policy_list.sort(key=cmp_to_key(_py3cmp))
return policy_list
def _py3cmp(a, b):
""" Python 2 can sort lists of mixed types. Strings < tuples. Without this function this fails on Python 3."""
try:
if a > b:
return 1
elif a < b:
return -1
else:
return 0
except TypeError as e:
# check to see if they're tuple-string
# always say strings are less than tuples (to maintain compatibility with python2)
str_ind = to_text(e).find('str')
tup_ind = to_text(e).find('tuple')
if -1 not in (str_ind, tup_ind):
if str_ind < tup_ind:
return -1
elif tup_ind < str_ind:
return 1
raise
def compare_policies(current_policy, new_policy, default_version="2008-10-17"):
""" Compares the existing policy and the updated policy
Returns True if there is a difference between policies.
"""
# https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_elements_version.html
if default_version:
if isinstance(current_policy, dict):
current_policy = current_policy.copy()
current_policy.setdefault("Version", default_version)
if isinstance(new_policy, dict):
new_policy = new_policy.copy()
new_policy.setdefault("Version", default_version)
return set(_hashable_policy(new_policy, [])) != set(_hashable_policy(current_policy, []))
def sort_json_policy_dict(policy_dict):
""" Sort any lists in an IAM JSON policy so that comparison of two policies with identical values but
different orders will return true
Args:
policy_dict (dict): Dict representing IAM JSON policy.
Basic Usage:
>>> my_iam_policy = {'Principle': {'AWS':["31","7","14","101"]}
>>> sort_json_policy_dict(my_iam_policy)
Returns:
Dict: Will return a copy of the policy as a Dict but any List will be sorted
{
'Principle': {
'AWS': [ '7', '14', '31', '101' ]
}
}
"""
def value_is_list(my_list):
checked_list = []
for item in my_list:
if isinstance(item, dict):
checked_list.append(sort_json_policy_dict(item))
elif isinstance(item, list):
checked_list.append(value_is_list(item))
else:
checked_list.append(item)
# Sort list. If it's a list of dictionaries, sort by tuple of key-value
# pairs, since Python 3 doesn't allow comparisons such as `<` between dictionaries.
checked_list.sort(key=lambda x: sorted(x.items()) if isinstance(x, dict) else x)
return checked_list
ordered_policy_dict = {}
for key, value in policy_dict.items():
if isinstance(value, dict):
ordered_policy_dict[key] = sort_json_policy_dict(value)
elif isinstance(value, list):
ordered_policy_dict[key] = value_is_list(value)
else:
ordered_policy_dict[key] = value
return ordered_policy_dict

View File

@@ -0,0 +1,387 @@
# Copyright: (c) 2018, Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from collections import namedtuple
from time import sleep
try:
from botocore.exceptions import BotoCoreError, ClientError, WaiterError
except ImportError:
pass
from ansible.module_utils._text import to_text
from ansible.module_utils.common.dict_transformations import snake_dict_to_camel_dict
from .ec2 import AWSRetry
from .ec2 import ansible_dict_to_boto3_tag_list
from .ec2 import boto3_tag_list_to_ansible_dict
from .ec2 import compare_aws_tags
from .waiters import get_waiter
Boto3ClientMethod = namedtuple('Boto3ClientMethod', ['name', 'waiter', 'operation_description', 'resource', 'retry_codes'])
# Whitelist boto3 client methods for cluster and instance resources
cluster_method_names = [
'create_db_cluster', 'restore_db_cluster_from_snapshot', 'restore_db_cluster_from_s3',
'restore_db_cluster_to_point_in_time', 'modify_db_cluster', 'delete_db_cluster', 'add_tags_to_resource',
'remove_tags_from_resource', 'list_tags_for_resource', 'promote_read_replica_db_cluster'
]
instance_method_names = [
'create_db_instance', 'restore_db_instance_to_point_in_time', 'restore_db_instance_from_s3',
'restore_db_instance_from_db_snapshot', 'create_db_instance_read_replica', 'modify_db_instance',
'delete_db_instance', 'add_tags_to_resource', 'remove_tags_from_resource', 'list_tags_for_resource',
'promote_read_replica', 'stop_db_instance', 'start_db_instance', 'reboot_db_instance', 'add_role_to_db_instance',
'remove_role_from_db_instance'
]
cluster_snapshot_method_names = [
'create_db_cluster_snapshot', 'delete_db_cluster_snapshot', 'add_tags_to_resource', 'remove_tags_from_resource',
'list_tags_for_resource', 'copy_db_cluster_snapshot'
]
instance_snapshot_method_names = [
'create_db_snapshot', 'delete_db_snapshot', 'add_tags_to_resource', 'remove_tags_from_resource',
'copy_db_snapshot', 'list_tags_for_resource'
]
def get_rds_method_attribute(method_name, module):
'''
Returns rds attributes of the specified method.
Parameters:
method_name (str): RDS method to call
module: AnsibleAWSModule
Returns:
Boto3ClientMethod (dict):
name (str): Name of method
waiter (str): Name of waiter associated with given method
operation_description (str): Description of method
resource (str): Type of resource this method applies to
One of ['instance', 'cluster', 'instance_snapshot', 'cluster_snapshot']
retry_codes (list): List of extra error codes to retry on
Raises:
NotImplementedError if wait is True but no waiter can be found for specified method
'''
waiter = ''
readable_op = method_name.replace('_', ' ').replace('db', 'DB')
resource = ''
retry_codes = []
if method_name in cluster_method_names and 'new_db_cluster_identifier' in module.params:
resource = 'cluster'
if method_name == 'delete_db_cluster':
waiter = 'cluster_deleted'
else:
waiter = 'cluster_available'
# Handle retry codes
if method_name == 'restore_db_cluster_from_snapshot':
retry_codes = ['InvalidDBClusterSnapshotState']
else:
retry_codes = ['InvalidDBClusterState']
elif method_name in instance_method_names and 'new_db_instance_identifier' in module.params:
resource = 'instance'
if method_name == 'delete_db_instance':
waiter = 'db_instance_deleted'
elif method_name == 'stop_db_instance':
waiter = 'db_instance_stopped'
elif method_name == 'add_role_to_db_instance':
waiter = 'role_associated'
elif method_name == 'remove_role_from_db_instance':
waiter = 'role_disassociated'
elif method_name == 'promote_read_replica':
waiter = 'read_replica_promoted'
else:
waiter = 'db_instance_available'
# Handle retry codes
if method_name == 'restore_db_instance_from_db_snapshot':
retry_codes = ['InvalidDBSnapshotState']
else:
retry_codes = ['InvalidDBInstanceState', 'InvalidDBSecurityGroupState']
elif method_name in cluster_snapshot_method_names and 'db_cluster_snapshot_identifier' in module.params:
resource = 'cluster_snapshot'
if method_name == 'delete_db_cluster_snapshot':
waiter = 'db_cluster_snapshot_deleted'
retry_codes = ['InvalidDBClusterSnapshotState']
elif method_name == 'create_db_cluster_snapshot':
waiter = 'db_cluster_snapshot_available'
retry_codes = ['InvalidDBClusterState']
else:
# Tagging
waiter = 'db_cluster_snapshot_available'
retry_codes = ['InvalidDBClusterSnapshotState']
elif method_name in instance_snapshot_method_names and 'db_snapshot_identifier' in module.params:
resource = 'instance_snapshot'
if method_name == 'delete_db_snapshot':
waiter = 'db_snapshot_deleted'
retry_codes = ['InvalidDBSnapshotState']
elif method_name == 'create_db_snapshot':
waiter = 'db_snapshot_available'
retry_codes = ['InvalidDBInstanceState']
else:
# Tagging
waiter = 'db_snapshot_available'
retry_codes = ['InvalidDBSnapshotState']
else:
if module.params.get('wait'):
raise NotImplementedError("method {0} hasn't been added to the list of accepted methods to use a waiter in module_utils/rds.py".format(method_name))
return Boto3ClientMethod(name=method_name, waiter=waiter, operation_description=readable_op,
resource=resource, retry_codes=retry_codes)
def get_final_identifier(method_name, module):
updated_identifier = None
apply_immediately = module.params.get('apply_immediately')
resource = get_rds_method_attribute(method_name, module).resource
if resource == 'cluster':
identifier = module.params['db_cluster_identifier']
updated_identifier = module.params['new_db_cluster_identifier']
elif resource == 'instance':
identifier = module.params['db_instance_identifier']
updated_identifier = module.params['new_db_instance_identifier']
elif resource == 'instance_snapshot':
identifier = module.params['db_snapshot_identifier']
elif resource == 'cluster_snapshot':
identifier = module.params['db_cluster_snapshot_identifier']
else:
raise NotImplementedError("method {0} hasn't been added to the list of accepted methods in module_utils/rds.py".format(method_name))
if not module.check_mode and updated_identifier and apply_immediately:
identifier = updated_identifier
return identifier
def handle_errors(module, exception, method_name, parameters):
if not isinstance(exception, ClientError):
module.fail_json_aws(exception, msg="Unexpected failure for method {0} with parameters {1}".format(method_name, parameters))
changed = True
error_code = exception.response['Error']['Code']
if (
method_name in ('modify_db_instance', 'modify_db_cluster') and
error_code == 'InvalidParameterCombination'
):
if 'No modifications were requested' in to_text(exception):
changed = False
elif 'ModifyDbCluster API' in to_text(exception):
module.fail_json_aws(exception, msg='It appears you are trying to modify attributes that are managed at the cluster level. Please see rds_cluster')
else:
module.fail_json_aws(exception, msg='Unable to {0}'.format(get_rds_method_attribute(method_name, module).operation_description))
elif method_name == 'promote_read_replica' and error_code == 'InvalidDBInstanceState':
if 'DB Instance is not a read replica' in to_text(exception):
changed = False
else:
module.fail_json_aws(exception, msg='Unable to {0}'.format(get_rds_method_attribute(method_name, module).operation_description))
elif method_name == 'promote_read_replica_db_cluster' and error_code == 'InvalidDBClusterStateFault':
if 'DB Cluster that is not a read replica' in to_text(exception):
changed = False
else:
module.fail_json_aws(exception, msg='Unable to {0}'.format(get_rds_method_attribute(method_name, module).operation_description))
elif method_name == 'create_db_cluster' and error_code == 'InvalidParameterValue':
accepted_engines = [
'aurora', 'aurora-mysql', 'aurora-postgresql'
]
if parameters.get('Engine') not in accepted_engines:
module.fail_json_aws(exception, msg='DB engine {0} should be one of {1}'.format(parameters.get('Engine'), accepted_engines))
else:
module.fail_json_aws(exception, msg='Unable to {0}'.format(get_rds_method_attribute(method_name, module).operation_description))
else:
module.fail_json_aws(exception, msg='Unable to {0}'.format(get_rds_method_attribute(method_name, module).operation_description))
return changed
def call_method(client, module, method_name, parameters):
result = {}
changed = True
if not module.check_mode:
wait = module.params.get('wait')
retry_codes = get_rds_method_attribute(method_name, module).retry_codes
method = getattr(client, method_name)
try:
result = AWSRetry.jittered_backoff(catch_extra_error_codes=retry_codes)(method)(**parameters)
except (BotoCoreError, ClientError) as e:
changed = handle_errors(module, e, method_name, parameters)
if wait and changed:
identifier = get_final_identifier(method_name, module)
wait_for_status(client, module, identifier, method_name)
return result, changed
def wait_for_instance_status(client, module, db_instance_id, waiter_name):
def wait(client, db_instance_id, waiter_name):
try:
waiter = client.get_waiter(waiter_name)
except ValueError:
# using a waiter in module_utils/waiters.py
waiter = get_waiter(client, waiter_name)
waiter.wait(WaiterConfig={'Delay': 60, 'MaxAttempts': 60}, DBInstanceIdentifier=db_instance_id)
waiter_expected_status = {
'db_instance_deleted': 'deleted',
'db_instance_stopped': 'stopped',
}
expected_status = waiter_expected_status.get(waiter_name, 'available')
for _wait_attempts in range(0, 10):
try:
wait(client, db_instance_id, waiter_name)
break
except WaiterError as e:
# Instance may be renamed and AWSRetry doesn't handle WaiterError
if e.last_response.get('Error', {}).get('Code') == 'DBInstanceNotFound':
sleep(10)
continue
module.fail_json_aws(e, msg='Error while waiting for DB instance {0} to be {1}'.format(db_instance_id, expected_status))
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg='Unexpected error while waiting for DB instance {0} to be {1}'.format(
db_instance_id, expected_status)
)
def wait_for_cluster_status(client, module, db_cluster_id, waiter_name):
try:
get_waiter(client, waiter_name).wait(DBClusterIdentifier=db_cluster_id)
except WaiterError as e:
if waiter_name == 'cluster_deleted':
msg = "Failed to wait for DB cluster {0} to be deleted".format(db_cluster_id)
else:
msg = "Failed to wait for DB cluster {0} to be available".format(db_cluster_id)
module.fail_json_aws(e, msg=msg)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Failed with an unexpected error while waiting for the DB cluster {0}".format(db_cluster_id))
def wait_for_instance_snapshot_status(client, module, db_snapshot_id, waiter_name):
try:
client.get_waiter(waiter_name).wait(DBSnapshotIdentifier=db_snapshot_id)
except WaiterError as e:
if waiter_name == 'db_snapshot_deleted':
msg = "Failed to wait for DB snapshot {0} to be deleted".format(db_snapshot_id)
else:
msg = "Failed to wait for DB snapshot {0} to be available".format(db_snapshot_id)
module.fail_json_aws(e, msg=msg)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Failed with an unexpected error while waiting for the DB snapshot {0}".format(db_snapshot_id))
def wait_for_cluster_snapshot_status(client, module, db_snapshot_id, waiter_name):
try:
client.get_waiter(waiter_name).wait(DBClusterSnapshotIdentifier=db_snapshot_id)
except WaiterError as e:
if waiter_name == 'db_cluster_snapshot_deleted':
msg = "Failed to wait for DB cluster snapshot {0} to be deleted".format(db_snapshot_id)
else:
msg = "Failed to wait for DB cluster snapshot {0} to be available".format(db_snapshot_id)
module.fail_json_aws(e, msg=msg)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Failed with an unexpected error while waiting for the DB cluster snapshot {0}".format(db_snapshot_id))
def wait_for_status(client, module, identifier, method_name):
rds_method_attributes = get_rds_method_attribute(method_name, module)
waiter_name = rds_method_attributes.waiter
resource = rds_method_attributes.resource
if resource == 'cluster':
wait_for_cluster_status(client, module, identifier, waiter_name)
elif resource == 'instance':
wait_for_instance_status(client, module, identifier, waiter_name)
elif resource == 'instance_snapshot':
wait_for_instance_snapshot_status(client, module, identifier, waiter_name)
elif resource == 'cluster_snapshot':
wait_for_cluster_snapshot_status(client, module, identifier, waiter_name)
def get_tags(client, module, resource_arn):
try:
return boto3_tag_list_to_ansible_dict(
client.list_tags_for_resource(ResourceName=resource_arn)['TagList']
)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Unable to describe tags")
def arg_spec_to_rds_params(options_dict):
tags = options_dict.pop('tags')
has_processor_features = False
if 'processor_features' in options_dict:
has_processor_features = True
processor_features = options_dict.pop('processor_features')
camel_options = snake_dict_to_camel_dict(options_dict, capitalize_first=True)
for key in list(camel_options.keys()):
for old, new in (('Db', 'DB'), ('Iam', 'IAM'), ('Az', 'AZ')):
if old in key:
camel_options[key.replace(old, new)] = camel_options.pop(key)
camel_options['Tags'] = tags
if has_processor_features:
camel_options['ProcessorFeatures'] = processor_features
return camel_options
def ensure_tags(client, module, resource_arn, existing_tags, tags, purge_tags):
if tags is None:
return False
tags_to_add, tags_to_remove = compare_aws_tags(existing_tags, tags, purge_tags)
changed = bool(tags_to_add or tags_to_remove)
if tags_to_add:
call_method(
client, module, method_name='add_tags_to_resource',
parameters={'ResourceName': resource_arn, 'Tags': ansible_dict_to_boto3_tag_list(tags_to_add)}
)
if tags_to_remove:
call_method(
client, module, method_name='remove_tags_from_resource',
parameters={'ResourceName': resource_arn, 'TagKeys': tags_to_remove}
)
return changed
def compare_iam_roles(existing_roles, target_roles, purge_roles):
'''
Returns differences between target and existing IAM roles
Parameters:
existing_roles (list): Existing IAM roles
target_roles (list): Target IAM roles
purge_roles (bool): Remove roles not in target_roles if True
Returns:
roles_to_add (list): List of IAM roles to add
roles_to_delete (list): List of IAM roles to delete
'''
existing_roles = [dict((k, v) for k, v in role.items() if k != 'status') for role in existing_roles]
roles_to_add = [role for role in target_roles if role not in existing_roles]
roles_to_remove = [role for role in existing_roles if role not in target_roles] if purge_roles else []
return roles_to_add, roles_to_remove
def update_iam_roles(client, module, instance_id, roles_to_add, roles_to_remove):
'''
Update a DB instance's associated IAM roles
Parameters:
client: RDS client
module: AnsibleAWSModule
instance_id (str): DB's instance ID
roles_to_add (list): List of IAM roles to add
roles_to_delete (list): List of IAM roles to delete
Returns:
changed (bool): True if changes were successfully made to DB instance's IAM roles; False if not
'''
for role in roles_to_remove:
params = {'DBInstanceIdentifier': instance_id,
'RoleArn': role['role_arn'],
'FeatureName': role['feature_name']}
_result, changed = call_method(client, module, method_name='remove_role_from_db_instance', parameters=params)
for role in roles_to_add:
params = {'DBInstanceIdentifier': instance_id,
'RoleArn': role['role_arn'],
'FeatureName': role['feature_name']}
_result, changed = call_method(client, module, method_name='add_role_to_db_instance', parameters=params)
return changed

View File

@@ -0,0 +1,78 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Michael DeHaan <michael.dehaan@gmail.com>, 2012-2013
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
try:
from botocore.exceptions import ClientError
HAS_BOTO3 = True
except ImportError:
HAS_BOTO3 = False
from .cloud import CloudRetry
def _botocore_exception_maybe():
"""
Allow for boto3 not being installed when using these utils by wrapping
botocore.exceptions instead of assigning from it directly.
"""
if HAS_BOTO3:
return ClientError
return type(None)
class AWSRetry(CloudRetry):
base_class = _botocore_exception_maybe()
@staticmethod
def status_code_from_exception(error):
return error.response['Error']['Code']
@staticmethod
def found(response_code, catch_extra_error_codes=None):
# This list of failures is based on this API Reference
# http://docs.aws.amazon.com/AWSEC2/latest/APIReference/errors-overview.html
#
# TooManyRequestsException comes from inside botocore when it
# does retrys, unfortunately however it does not try long
# enough to allow some services such as API Gateway to
# complete configuration. At the moment of writing there is a
# botocore/boto3 bug open to fix this.
#
# https://github.com/boto/boto3/issues/876 (and linked PRs etc)
retry_on = [
'RequestLimitExceeded', 'Unavailable', 'ServiceUnavailable',
'InternalFailure', 'InternalError', 'TooManyRequestsException',
'Throttling'
]
if catch_extra_error_codes:
retry_on.extend(catch_extra_error_codes)
return response_code in retry_on

View File

@@ -0,0 +1,64 @@
# This file is part of Ansible
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
try:
import botocore
except ImportError:
pass # caught by AnsibleAWSModule
from ansible_collections.amazon.aws.plugins.module_utils.core import is_boto3_error_code
from ansible_collections.amazon.aws.plugins.module_utils.tagging import ansible_dict_to_boto3_tag_list
from ansible_collections.amazon.aws.plugins.module_utils.tagging import boto3_tag_list_to_ansible_dict
from ansible_collections.amazon.aws.plugins.module_utils.tagging import compare_aws_tags
def manage_tags(module, client, resource_type, resource_id, new_tags, purge_tags):
if new_tags is None:
return False
old_tags = get_tags(module, client, resource_type, resource_id)
tags_to_set, tags_to_delete = compare_aws_tags(old_tags, new_tags, purge_tags=purge_tags)
change_params = dict()
if tags_to_set:
change_params['AddTags'] = ansible_dict_to_boto3_tag_list(tags_to_set)
if tags_to_delete:
change_params['RemoveTagKeys'] = tags_to_delete
if not change_params:
return False
if module.check_mode:
return True
try:
client.change_tags_for_resource(
ResourceType=resource_type,
ResourceId=resource_id,
**change_params
)
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e:
module.fail_json_aws(e, msg='Failed to update tags on {0}'.format(resource_type),
resource_id=resource_id, change_params=change_params)
return True
def get_tags(module, client, resource_type, resource_id):
try:
tagset = client.list_tags_for_resource(
ResourceType=resource_type,
ResourceId=resource_id,
)
except is_boto3_error_code('NoSuchHealthCheck'):
return {}
except is_boto3_error_code('NoSuchHostedZone'): # pylint: disable=duplicate-except
return {}
except (botocore.exceptions.BotoCoreError, botocore.exceptions.ClientError) as e: # pylint: disable=duplicate-except
module.fail_json_aws(e, msg='Failed to fetch tags on {0}'.format(resource_type),
resource_id=resource_id)
tags = boto3_tag_list_to_ansible_dict(tagset['ResourceTagSet']['Tags'])
return tags

View File

@@ -0,0 +1,102 @@
# Copyright (c) 2018 Red Hat, Inc.
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
try:
from botocore.exceptions import BotoCoreError, ClientError
except ImportError:
pass # Handled by the calling module
HAS_MD5 = True
try:
from hashlib import md5
except ImportError:
try:
from md5 import md5
except ImportError:
HAS_MD5 = False
import string
def calculate_etag(module, filename, etag, s3, bucket, obj, version=None):
if not HAS_MD5:
return None
if '-' in etag:
# Multi-part ETag; a hash of the hashes of each part.
parts = int(etag[1:-1].split('-')[1])
digests = []
s3_kwargs = dict(
Bucket=bucket,
Key=obj,
)
if version:
s3_kwargs['VersionId'] = version
with open(filename, 'rb') as f:
for part_num in range(1, parts + 1):
s3_kwargs['PartNumber'] = part_num
try:
head = s3.head_object(**s3_kwargs)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Failed to get head object")
digests.append(md5(f.read(int(head['ContentLength']))))
digest_squared = md5(b''.join(m.digest() for m in digests))
return '"{0}-{1}"'.format(digest_squared.hexdigest(), len(digests))
else: # Compute the MD5 sum normally
return '"{0}"'.format(module.md5(filename))
def calculate_etag_content(module, content, etag, s3, bucket, obj, version=None):
if not HAS_MD5:
return None
if '-' in etag:
# Multi-part ETag; a hash of the hashes of each part.
parts = int(etag[1:-1].split('-')[1])
digests = []
offset = 0
s3_kwargs = dict(
Bucket=bucket,
Key=obj,
)
if version:
s3_kwargs['VersionId'] = version
for part_num in range(1, parts + 1):
s3_kwargs['PartNumber'] = part_num
try:
head = s3.head_object(**s3_kwargs)
except (BotoCoreError, ClientError) as e:
module.fail_json_aws(e, msg="Failed to get head object")
length = int(head['ContentLength'])
digests.append(md5(content[offset:offset + length]))
offset += length
digest_squared = md5(b''.join(m.digest() for m in digests))
return '"{0}-{1}"'.format(digest_squared.hexdigest(), len(digests))
else: # Compute the MD5 sum normally
return '"{0}"'.format(md5(content).hexdigest())
def validate_bucket_name(module, name):
# See: https://docs.aws.amazon.com/AmazonS3/latest/userguide/bucketnamingrules.html
if len(name) < 3:
module.fail_json(msg='the length of an S3 bucket must be at least 3 characters')
if len(name) > 63:
module.fail_json(msg='the length of an S3 bucket cannot exceed 63 characters')
legal_characters = string.ascii_lowercase + ".-" + string.digits
illegal_characters = [c for c in name if c not in legal_characters]
if illegal_characters:
module.fail_json(msg='invalid character(s) found in the bucket name')
if name[-1] not in string.ascii_lowercase + string.digits:
module.fail_json(msg='bucket names must begin and end with a letter or number')
return True

View File

@@ -0,0 +1,181 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Michael DeHaan <michael.dehaan@gmail.com>, 2012-2013
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from ansible.module_utils._text import to_native
from ansible.module_utils._text import to_text
from ansible.module_utils.six import string_types
def boto3_tag_list_to_ansible_dict(tags_list, tag_name_key_name=None, tag_value_key_name=None):
""" Convert a boto3 list of resource tags to a flat dict of key:value pairs
Args:
tags_list (list): List of dicts representing AWS tags.
tag_name_key_name (str): Value to use as the key for all tag keys (useful because boto3 doesn't always use "Key")
tag_value_key_name (str): Value to use as the key for all tag values (useful because boto3 doesn't always use "Value")
Basic Usage:
>>> tags_list = [{'Key': 'MyTagKey', 'Value': 'MyTagValue'}]
>>> boto3_tag_list_to_ansible_dict(tags_list)
[
{
'Key': 'MyTagKey',
'Value': 'MyTagValue'
}
]
Returns:
Dict: Dict of key:value pairs representing AWS tags
{
'MyTagKey': 'MyTagValue',
}
"""
if tag_name_key_name and tag_value_key_name:
tag_candidates = {tag_name_key_name: tag_value_key_name}
else:
tag_candidates = {'key': 'value', 'Key': 'Value'}
# minio seems to return [{}] as an empty tags_list
if not tags_list or not any(tag for tag in tags_list):
return {}
for k, v in tag_candidates.items():
if k in tags_list[0] and v in tags_list[0]:
return dict((tag[k], tag[v]) for tag in tags_list)
raise ValueError("Couldn't find tag key (candidates %s) in tag list %s" % (str(tag_candidates), str(tags_list)))
def ansible_dict_to_boto3_tag_list(tags_dict, tag_name_key_name='Key', tag_value_key_name='Value'):
""" Convert a flat dict of key:value pairs representing AWS resource tags to a boto3 list of dicts
Args:
tags_dict (dict): Dict representing AWS resource tags.
tag_name_key_name (str): Value to use as the key for all tag keys (useful because boto3 doesn't always use "Key")
tag_value_key_name (str): Value to use as the key for all tag values (useful because boto3 doesn't always use "Value")
Basic Usage:
>>> tags_dict = {'MyTagKey': 'MyTagValue'}
>>> ansible_dict_to_boto3_tag_list(tags_dict)
{
'MyTagKey': 'MyTagValue'
}
Returns:
List: List of dicts containing tag keys and values
[
{
'Key': 'MyTagKey',
'Value': 'MyTagValue'
}
]
"""
if not tags_dict:
return []
tags_list = []
for k, v in tags_dict.items():
tags_list.append({tag_name_key_name: k, tag_value_key_name: to_native(v)})
return tags_list
def boto3_tag_specifications(tags_dict, types=None):
""" Converts a list of resource types and a flat dictionary of key:value pairs representing AWS
resource tags to a TagSpecification object.
https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_TagSpecification.html
Args:
tags_dict (dict): Dict representing AWS resource tags.
types (list) A list of resource types to be tagged.
Basic Usage:
>>> tags_dict = {'MyTagKey': 'MyTagValue'}
>>> boto3_tag_specifications(tags_dict, ['instance'])
[
{
'ResourceType': 'instance',
'Tags': [
{
'Key': 'MyTagKey',
'Value': 'MyTagValue'
}
]
}
]
Returns:
List: List of dictionaries representing an AWS Tag Specification
"""
if not tags_dict:
return None
specifications = list()
tag_list = ansible_dict_to_boto3_tag_list(tags_dict)
if not types:
specifications.append(dict(Tags=tag_list))
return specifications
if isinstance(types, string_types):
types = [types]
for type_name in types:
specifications.append(dict(ResourceType=type_name, Tags=tag_list))
return specifications
def compare_aws_tags(current_tags_dict, new_tags_dict, purge_tags=True):
"""
Compare two dicts of AWS tags. Dicts are expected to of been created using 'boto3_tag_list_to_ansible_dict' helper function.
Two dicts are returned - the first is tags to be set, the second is any tags to remove. Since the AWS APIs differ
these may not be able to be used out of the box.
:param current_tags_dict:
:param new_tags_dict:
:param purge_tags:
:return: tag_key_value_pairs_to_set: a dict of key value pairs that need to be set in AWS. If all tags are identical this dict will be empty
:return: tag_keys_to_unset: a list of key names (type str) that need to be unset in AWS. If no tags need to be unset this list will be empty
"""
tag_key_value_pairs_to_set = {}
tag_keys_to_unset = []
if purge_tags:
for key in current_tags_dict.keys():
if key in new_tags_dict:
continue
# Amazon have reserved 'aws:*' tags, we should avoid purging them as
# this probably isn't what people want to do...
if key.startswith('aws:'):
continue
tag_keys_to_unset.append(key)
for key in set(new_tags_dict.keys()) - set(tag_keys_to_unset):
if to_text(new_tags_dict[key]) != current_tags_dict.get(key):
tag_key_value_pairs_to_set[key] = new_tags_dict[key]
return tag_key_value_pairs_to_set, tag_keys_to_unset

View File

@@ -0,0 +1,83 @@
# Copyright: Ansible Project
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import absolute_import, division, print_function
__metaclass__ = type
import string
import textwrap
from ansible.module_utils._text import to_native
from ansible.module_utils.six.moves.urllib import parse as urlparse
def _windows_callback_script(passwd=None):
script_url = 'https://raw.githubusercontent.com/ansible/ansible/devel/examples/scripts/ConfigureRemotingForAnsible.ps1'
if passwd is not None:
passwd = passwd.replace("'", "''")
script_tpl = """\
<powershell>
$admin = [adsi]('WinNT://./administrator, user')
$admin.PSBase.Invoke('SetPassword', '${PASS}')
Invoke-Expression ((New-Object System.Net.Webclient).DownloadString('${SCRIPT}'))
</powershell>
"""
else:
script_tpl = """\
<powershell>
$admin = [adsi]('WinNT://./administrator, user')
Invoke-Expression ((New-Object System.Net.Webclient).DownloadString('${SCRIPT}'))
</powershell>
"""
tpl = string.Template(textwrap.dedent(script_tpl))
return tpl.safe_substitute(PASS=passwd, SCRIPT=script_url)
def _linux_callback_script(tower_address, template_id, host_config_key):
template_id = urlparse.quote(template_id)
tower_address = urlparse.quote(tower_address)
host_config_key = host_config_key.replace("'", "'\"'\"'")
script_tpl = """\
#!/bin/bash
set -x
retry_attempts=10
attempt=0
while [[ $attempt -lt $retry_attempts ]]
do
status_code=$(curl --max-time 10 -v -k -s -i \
--data 'host_config_key=${host_config_key}' \
'https://${tower_address}/api/v2/job_templates/${template_id}/callback/' \
| head -n 1 \
| awk '{print $2}')
if [[ $status_code == 404 ]]
then
status_code=$(curl --max-time 10 -v -k -s -i \
--data 'host_config_key=${host_config_key}' \
'https://${tower_address}/api/v1/job_templates/${template_id}/callback/' \
| head -n 1 \
| awk '{print $2}')
# fall back to using V1 API for Tower 3.1 and below, since v2 API will always 404
fi
if [[ $status_code == 201 ]]
then
exit 0
fi
attempt=$(( attempt + 1 ))
echo "$${status_code} received... retrying in 1 minute. (Attempt $${attempt})"
sleep 60
done
exit 1
"""
tpl = string.Template(textwrap.dedent(script_tpl))
return tpl.safe_substitute(tower_address=tower_address,
template_id=template_id,
host_config_key=host_config_key)
def tower_callback_script(tower_address, job_template_id, host_config_key, windows, passwd):
if windows:
return to_native(_windows_callback_script(passwd=passwd))
return _linux_callback_script(tower_address, job_template_id, host_config_key)

View File

@@ -0,0 +1,140 @@
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Copyright (c), Michael DeHaan <michael.dehaan@gmail.com>, 2012-2013
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
from ansible.module_utils.six import string_types
from ansible.module_utils.six import integer_types
def ansible_dict_to_boto3_filter_list(filters_dict):
""" Convert an Ansible dict of filters to list of dicts that boto3 can use
Args:
filters_dict (dict): Dict of AWS filters.
Basic Usage:
>>> filters = {'some-aws-id': 'i-01234567'}
>>> ansible_dict_to_boto3_filter_list(filters)
{
'some-aws-id': 'i-01234567'
}
Returns:
List: List of AWS filters and their values
[
{
'Name': 'some-aws-id',
'Values': [
'i-01234567',
]
}
]
"""
filters_list = []
for k, v in filters_dict.items():
filter_dict = {'Name': k}
if isinstance(v, bool):
filter_dict['Values'] = [str(v).lower()]
elif isinstance(v, integer_types):
filter_dict['Values'] = [str(v)]
elif isinstance(v, string_types):
filter_dict['Values'] = [v]
else:
filter_dict['Values'] = v
filters_list.append(filter_dict)
return filters_list
def map_complex_type(complex_type, type_map):
"""
Allows to cast elements within a dictionary to a specific type
Example of usage:
DEPLOYMENT_CONFIGURATION_TYPE_MAP = {
'maximum_percent': 'int',
'minimum_healthy_percent': 'int'
}
deployment_configuration = map_complex_type(module.params['deployment_configuration'],
DEPLOYMENT_CONFIGURATION_TYPE_MAP)
This ensures all keys within the root element are casted and valid integers
"""
if complex_type is None:
return
new_type = type(complex_type)()
if isinstance(complex_type, dict):
for key in complex_type:
if key in type_map:
if isinstance(type_map[key], list):
new_type[key] = map_complex_type(
complex_type[key],
type_map[key][0])
else:
new_type[key] = map_complex_type(
complex_type[key],
type_map[key])
else:
new_type[key] = complex_type[key]
elif isinstance(complex_type, list):
for i in range(len(complex_type)):
new_type.append(map_complex_type(
complex_type[i],
type_map))
elif type_map:
return globals()['__builtins__'][type_map](complex_type)
return new_type
def scrub_none_parameters(parameters, descend_into_lists=True):
"""
Iterate over a dictionary removing any keys that have a None value
Reference: https://github.com/ansible-collections/community.aws/issues/251
Credit: https://medium.com/better-programming/how-to-remove-null-none-values-from-a-dictionary-in-python-1bedf1aab5e4
:param descend_into_lists: whether or not to descend in to lists to continue to remove None values
:param parameters: parameter dict
:return: parameter dict with all keys = None removed
"""
clean_parameters = {}
for k, v in parameters.items():
if isinstance(v, dict):
clean_parameters[k] = scrub_none_parameters(v, descend_into_lists=descend_into_lists)
elif descend_into_lists and isinstance(v, list):
clean_parameters[k] = [scrub_none_parameters(vv, descend_into_lists=descend_into_lists) if isinstance(vv, dict) else vv for vv in v]
elif v is not None:
clean_parameters[k] = v
return clean_parameters

View File

@@ -0,0 +1,238 @@
# Copyright: (c) 2018, Aaron Haaf <aabonh@gmail.com>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
import datetime
import hashlib
import hmac
import operator
try:
from boto3 import session
except ImportError:
pass
from ansible.module_utils.six.moves.urllib.parse import urlencode
from ansible.module_utils.urls import open_url
from .ec2 import HAS_BOTO3
from .ec2 import get_aws_connection_info
import ansible.module_utils.common.warnings as ansible_warnings
def hexdigest(s):
"""
Returns the sha256 hexdigest of a string after encoding.
"""
ansible_warnings.deprecate(
'amazon.aws.module_utils.urls.hexdigest is unused and has been deprecated.',
version='7.0.0', collection_name='amazon.aws')
return hashlib.sha256(s.encode("utf-8")).hexdigest()
def format_querystring(params=None):
"""
Returns properly url-encoded query string from the provided params dict.
It's specially sorted for cannonical requests
"""
ansible_warnings.deprecate(
'amazon.aws.module_utils.urls.format_querystring is unused and has been deprecated.',
version='7.0.0', collection_name='amazon.aws')
if not params:
return ""
# Query string values must be URL-encoded (space=%20). The parameters must be sorted by name.
return urlencode(sorted(params.items(), operator.itemgetter(0)))
# Key derivation functions. See:
# http://docs.aws.amazon.com/general/latest/gr/signature-v4-examples.html#signature-v4-examples-python
def sign(key, msg):
'''
Return digest for key applied to msg
'''
ansible_warnings.deprecate(
'amazon.aws.module_utils.urls.sign is unused and has been deprecated.',
version='7.0.0', collection_name='amazon.aws')
return hmac.new(key, msg.encode("utf-8"), hashlib.sha256).digest()
def get_signature_key(key, dateStamp, regionName, serviceName):
'''
Returns signature key for AWS resource
'''
ansible_warnings.deprecate(
'amazon.aws.module_utils.urls.get_signature_key is unused and has been deprecated.',
version='7.0.0', collection_name='amazon.aws')
kDate = sign(("AWS4" + key).encode("utf-8"), dateStamp)
kRegion = sign(kDate, regionName)
kService = sign(kRegion, serviceName)
kSigning = sign(kService, "aws4_request")
return kSigning
def get_aws_credentials_object(module):
'''
Returns aws_access_key_id, aws_secret_access_key, session_token for a module.
'''
ansible_warnings.deprecate(
'amazon.aws.module_utils.urls.get_aws_credentials_object is unused and has been deprecated.',
version='7.0.0', collection_name='amazon.aws')
if not HAS_BOTO3:
module.fail_json("get_aws_credentials_object requires boto3")
dummy, dummy, boto_params = get_aws_connection_info(module, boto3=True)
s = session.Session(**boto_params)
return s.get_credentials()
# Reference: https://docs.aws.amazon.com/general/latest/gr/sigv4-signed-request-examples.html
def signed_request(
module=None,
method="GET", service=None, host=None, uri=None,
query=None, body="", headers=None,
session_in_header=True, session_in_query=False
):
"""Generate a SigV4 request to an AWS resource for a module
This is used if you wish to authenticate with AWS credentials to a secure endpoint like an elastisearch domain.
Returns :class:`HTTPResponse` object.
Example:
result = signed_request(
module=this,
service="es",
host="search-recipes1-xxxxxxxxx.us-west-2.es.amazonaws.com",
)
:kwarg host: endpoint to talk to
:kwarg service: AWS id of service (like `ec2` or `es`)
:kwarg module: An AnsibleAWSModule to gather connection info from
:kwarg body: (optional) Payload to send
:kwarg method: (optional) HTTP verb to use
:kwarg query: (optional) dict of query params to handle
:kwarg uri: (optional) Resource path without query parameters
:kwarg session_in_header: (optional) Add the session token to the headers
:kwarg session_in_query: (optional) Add the session token to the query parameters
:returns: HTTPResponse
"""
module.deprecate(
'amazon.aws.module_utils.urls.signed_request is unused and has been deprecated.',
version='7.0.0', collection_name='amazon.aws')
if not HAS_BOTO3:
module.fail_json("A sigv4 signed_request requires boto3")
# "Constants"
t = datetime.datetime.utcnow()
amz_date = t.strftime("%Y%m%dT%H%M%SZ")
datestamp = t.strftime("%Y%m%d") # Date w/o time, used in credential scope
algorithm = "AWS4-HMAC-SHA256"
# AWS stuff
region, dummy, dummy = get_aws_connection_info(module, boto3=True)
credentials = get_aws_credentials_object(module)
access_key = credentials.access_key
secret_key = credentials.secret_key
session_token = credentials.token
if not access_key:
module.fail_json(msg="aws_access_key_id is missing")
if not secret_key:
module.fail_json(msg="aws_secret_access_key is missing")
credential_scope = "/".join([datestamp, region, service, "aws4_request"])
# Argument Defaults
uri = uri or "/"
query_string = format_querystring(query) if query else ""
headers = headers or dict()
query = query or dict()
headers.update({
"host": host,
"x-amz-date": amz_date,
})
# Handle adding of session_token if present
if session_token:
if session_in_header:
headers["X-Amz-Security-Token"] = session_token
if session_in_query:
query["X-Amz-Security-Token"] = session_token
if method == "GET":
body = ""
# Derived data
body_hash = hexdigest(body)
signed_headers = ";".join(sorted(headers.keys()))
# Setup Cannonical request to generate auth token
cannonical_headers = "\n".join([
key.lower().strip() + ":" + value for key, value in headers.items()
]) + "\n" # Note additional trailing newline
cannonical_request = "\n".join([
method,
uri,
query_string,
cannonical_headers,
signed_headers,
body_hash,
])
string_to_sign = "\n".join([algorithm, amz_date, credential_scope, hexdigest(cannonical_request)])
# Sign the Cannonical request
signing_key = get_signature_key(secret_key, datestamp, region, service)
signature = hmac.new(signing_key, string_to_sign.encode("utf-8"), hashlib.sha256).hexdigest()
# Make auth header with that info
authorization_header = "{0} Credential={1}/{2}, SignedHeaders={3}, Signature={4}".format(
algorithm, access_key, credential_scope, signed_headers, signature
)
# PERFORM THE REQUEST!
url = "https://" + host + uri
if query_string != "":
url = url + "?" + query_string
final_headers = {
"x-amz-date": amz_date,
"Authorization": authorization_header,
}
final_headers.update(headers)
return open_url(url, method=method, data=body, headers=final_headers)

View File

@@ -0,0 +1,18 @@
# -*- coding: utf-8 -*-
# Copyright: (c) 2021, Felix Fontein <felix@fontein.de>
# GNU General Public License v3.0+ (see COPYING or https://www.gnu.org/licenses/gpl-3.0.txt)
"""Provide version object to compare version numbers."""
from __future__ import absolute_import, division, print_function
__metaclass__ = type
# Once we drop support for Ansible 2.9, ansible-base 2.10, and ansible-core 2.11, we can
# remove the _version.py file, and replace the following import by
#
# from ansible.module_utils.compat.version import LooseVersion
from ._version import LooseVersion # pylint: disable=unused-import

View File

@@ -0,0 +1,224 @@
# Copyright (c) 2017 Will Thames
#
# This code is part of Ansible, but is an independent component.
# This particular file snippet, and this file snippet only, is BSD licensed.
# Modules you write using this snippet, which is embedded dynamically by Ansible
# still belong to the author of the module, and may assign their own license
# to the complete work.
#
# Redistribution and use in source and binary forms, with or without modification,
# are permitted provided that the following conditions are met:
#
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright notice,
# this list of conditions and the following disclaimer in the documentation
# and/or other materials provided with the distribution.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED.
# IN NO EVENT SHALL THE COPYRIGHT HOLDER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT,
# INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS
# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
# LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE
# USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
#
"""
This module adds shared support for Web Application Firewall modules
"""
from __future__ import (absolute_import, division, print_function)
__metaclass__ = type
try:
import botocore
except ImportError:
pass # caught by imported HAS_BOTO3
from ansible.module_utils.common.dict_transformations import camel_dict_to_snake_dict
from .ec2 import AWSRetry
from .waiters import get_waiter
MATCH_LOOKUP = {
'byte': {
'method': 'byte_match_set',
'conditionset': 'ByteMatchSet',
'conditiontuple': 'ByteMatchTuple',
'type': 'ByteMatch'
},
'geo': {
'method': 'geo_match_set',
'conditionset': 'GeoMatchSet',
'conditiontuple': 'GeoMatchConstraint',
'type': 'GeoMatch'
},
'ip': {
'method': 'ip_set',
'conditionset': 'IPSet',
'conditiontuple': 'IPSetDescriptor',
'type': 'IPMatch'
},
'regex': {
'method': 'regex_match_set',
'conditionset': 'RegexMatchSet',
'conditiontuple': 'RegexMatchTuple',
'type': 'RegexMatch'
},
'size': {
'method': 'size_constraint_set',
'conditionset': 'SizeConstraintSet',
'conditiontuple': 'SizeConstraint',
'type': 'SizeConstraint'
},
'sql': {
'method': 'sql_injection_match_set',
'conditionset': 'SqlInjectionMatchSet',
'conditiontuple': 'SqlInjectionMatchTuple',
'type': 'SqlInjectionMatch',
},
'xss': {
'method': 'xss_match_set',
'conditionset': 'XssMatchSet',
'conditiontuple': 'XssMatchTuple',
'type': 'XssMatch'
},
}
@AWSRetry.jittered_backoff(delay=5)
def get_rule_with_backoff(client, rule_id):
return client.get_rule(RuleId=rule_id)['Rule']
@AWSRetry.jittered_backoff(delay=5)
def get_byte_match_set_with_backoff(client, byte_match_set_id):
return client.get_byte_match_set(ByteMatchSetId=byte_match_set_id)['ByteMatchSet']
@AWSRetry.jittered_backoff(delay=5)
def get_ip_set_with_backoff(client, ip_set_id):
return client.get_ip_set(IPSetId=ip_set_id)['IPSet']
@AWSRetry.jittered_backoff(delay=5)
def get_size_constraint_set_with_backoff(client, size_constraint_set_id):
return client.get_size_constraint_set(SizeConstraintSetId=size_constraint_set_id)['SizeConstraintSet']
@AWSRetry.jittered_backoff(delay=5)
def get_sql_injection_match_set_with_backoff(client, sql_injection_match_set_id):
return client.get_sql_injection_match_set(SqlInjectionMatchSetId=sql_injection_match_set_id)['SqlInjectionMatchSet']
@AWSRetry.jittered_backoff(delay=5)
def get_xss_match_set_with_backoff(client, xss_match_set_id):
return client.get_xss_match_set(XssMatchSetId=xss_match_set_id)['XssMatchSet']
def get_rule(client, module, rule_id):
try:
rule = get_rule_with_backoff(client, rule_id)
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
module.fail_json_aws(e, msg="Couldn't obtain waf rule")
match_sets = {
'ByteMatch': get_byte_match_set_with_backoff,
'IPMatch': get_ip_set_with_backoff,
'SizeConstraint': get_size_constraint_set_with_backoff,
'SqlInjectionMatch': get_sql_injection_match_set_with_backoff,
'XssMatch': get_xss_match_set_with_backoff
}
if 'Predicates' in rule:
for predicate in rule['Predicates']:
if predicate['Type'] in match_sets:
predicate.update(match_sets[predicate['Type']](client, predicate['DataId']))
# replaced by Id from the relevant MatchSet
del predicate['DataId']
return rule
@AWSRetry.jittered_backoff(delay=5)
def get_web_acl_with_backoff(client, web_acl_id):
return client.get_web_acl(WebACLId=web_acl_id)['WebACL']
def get_web_acl(client, module, web_acl_id):
try:
web_acl = get_web_acl_with_backoff(client, web_acl_id)
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
module.fail_json_aws(e, msg="Couldn't obtain web acl")
if web_acl:
try:
for rule in web_acl['Rules']:
rule.update(get_rule(client, module, rule['RuleId']))
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
module.fail_json_aws(e, msg="Couldn't obtain web acl rule")
return camel_dict_to_snake_dict(web_acl)
@AWSRetry.jittered_backoff(delay=5)
def list_rules_with_backoff(client):
paginator = client.get_paginator('list_rules')
return paginator.paginate().build_full_result()['Rules']
@AWSRetry.jittered_backoff(delay=5)
def list_regional_rules_with_backoff(client):
resp = client.list_rules()
rules = []
while resp:
rules += resp['Rules']
resp = client.list_rules(NextMarker=resp['NextMarker']) if 'NextMarker' in resp else None
return rules
@AWSRetry.jittered_backoff(delay=5)
def list_web_acls_with_backoff(client):
paginator = client.get_paginator('list_web_acls')
return paginator.paginate().build_full_result()['WebACLs']
@AWSRetry.jittered_backoff(delay=5)
def list_regional_web_acls_with_backoff(client):
resp = client.list_web_acls()
acls = []
while resp:
acls += resp['WebACLs']
resp = client.list_web_acls(NextMarker=resp['NextMarker']) if 'NextMarker' in resp else None
return acls
def list_web_acls(client, module):
try:
if client.__class__.__name__ == 'WAF':
return list_web_acls_with_backoff(client)
elif client.__class__.__name__ == 'WAFRegional':
return list_regional_web_acls_with_backoff(client)
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
module.fail_json_aws(e, msg="Couldn't obtain web acls")
def get_change_token(client, module):
try:
token = client.get_change_token()
return token['ChangeToken']
except (botocore.exceptions.ClientError, botocore.exceptions.BotoCoreError) as e:
module.fail_json_aws(e, msg="Couldn't obtain change token")
@AWSRetry.jittered_backoff(backoff=2, catch_extra_error_codes=['WAFStaleDataException'])
def run_func_with_change_token_backoff(client, module, params, func, wait=False):
params['ChangeToken'] = get_change_token(client, module)
result = func(**params)
if wait:
get_waiter(
client, 'change_token_in_sync',
).wait(
ChangeToken=result['ChangeToken']
)
return result

Some files were not shown because too many files have changed in this diff Show More