×
Home About Blog Contact AiDOOS Model Knowledge Base

Welcome to Knowledge Base!

KB at your finger tips

This is one stop global knowledge base where you can learn about all the products, solutions and support features.

Categories
All
CRM
Salesforce
ERP
SAP
Storage and Backups
Purestorage Netapp Nutanix
Cloud
AWS
Computing
IBM
Database
MongoDB MySQL PostgreSQL
DevOps
Docker Kubernetes Jenkins GitLab
Mobile
React Native Android
Web
React Angular PHP
Programming
Python
Cloud-AWS
Use Amazon EMR for processing data

How can I use Amazon EMR to process data?

Last updated: 2022-11-17

I want to use Amazon EMR to process data.

Resolution

Amazon EMR processes data using Amazon Elastic Compute Cloud (Amazon EC2) instances and open-source applications such as Apache Spark, HBase, Presto, and Flink.

To launch your first EMR cluster, follow the video tutorial in the article, or see Tutorial: Getting started with Amazon EMR.

Overview of Amazon EMR

Overview of Amazon EMR architecture

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support
Overview of Amazon Mechanical Turk

What is Amazon Mechanical Turk?

Last updated: 2022-10-18

What is Amazon Mechanical Turk?

Resolution

Amazon Mechanical Turk is a forum where Requesters post work as Human Intelligence Tasks (HITs). Workers complete HITs in exchange for a reward. You write, test, and publish your HIT using the Mechanical Turk developer sandbox, Amazon Mechanical Turk APIs, and AWS SDKs.

Here are some common tasks posted by Requesters:

  • Localization and transcription services
  • Audio editing
  • Information gathering tasks (surveys)
  • Machine learning
  • Information gathering
  • Photo and video processing
  • Data collection

To set up your AWS account and develop solutions using the Mechanical Turk web service, see Setting up accounts and tools. For other common scenarios and resources, see Common use scenarios.

Amazon Mechanical Turk

Amazon Mechanical Turk FAQs

Amazon Mechanical Turk Worker FAQs

Implementing Amazon Mechanical Turk

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support
Read article
Stop and start instances using the AWS Instance Scheduler with CloudFormation

How do I use Instance Scheduler with CloudFormation to schedule EC2 instances?

Last updated: 2022-10-28

I want to use AWS Instance Scheduler with AWS CloudFormation to schedule Amazon Elastic Compute Cloud (Amazon EC2) instances.

Short description

Use CloudFormation templates to automate the deployment of AWS Instance Scheduler.

Note: Currently, you can't use the templates in the Asia Pacific (Jakarta) and Asia Pacific (Osaka) AWS Regions.

Important: If you use the Instance Scheduler for EC2 instances with an encrypted Amazon Elastic Block Store (Amazon EBS), then your instances can't be started. To start your instances, you must grant the Instance Scheduler a key user role with a key policy to encrypt or decrypt EBS volumes. You must add the key policy to the AWS Key Management Service (AWS KMS) key to allow the key user role to use this key.

Resolution

Install the Instance Scheduler command line interface (CLI).

To verify that the installation is successful, run the following command: 

$ scheduler-cli --version

Create a CloudFormation stack with the Instance Scheduler template

The stack deploys an AWS Lambda function, an Amazon DynamoDB table, an Amazon EventBridge rule, and Amazon CloudWatch custom metrics.

  1. Open the AWS Management Console.
  2. Open CloudFormation with the Instance Scheduler template. Or, go to the Step 1. Launch the instance scheduler stack page, and choose Launch Solution .
    Note: The template is launched in the US East (N. Virginia) Region by default.
  3. In the navigation bar, select the AWS Region where you want to launch your stack with the template, and then choose Next .
  4. For Stack name , name your stack.
  5. For Instance Scheduler TagName , you can keep the default value as Schedule , or customize it.
  6. For Frequency , choose a frequency in minutes to run your scheduler. For example, you can choose 5 minutes.
    Note: The frequency is the number of minutes that pass before EventBridge initiates the Lambda function again for the Instance Scheduler. If you have a large number of instances, then use the highest frequency possible to avoid throttling. If the frequency isn't often enough for your needs, then you can adjust the Frequency property later.
  7. For Enable CloudWatch Logs , choose Yes .
  8. For Started tags , enter state=started .
  9. For Stopped tags , enter state=stopped .
  10. For cross-account scheduling, provide the Cross-account roles parameter. Put in the ARNs for every role from the secondary accounts, separated by commas. If you aren't using cross-account scheduling, then leave the parameter empty.
  11. For all other parameters, customize the stack for your needs.
  12. Choose Next .
  13. On the Options page, choose Next .
  14. Review your settings, and then select I acknowledge that AWS CloudFormation might create IAM resources .
  15. Choose Create .

Create the periods

To create periods, you can use the Instance Scheduler CLI, DynamoDB console, or Custom resources. For more information on time periods, see Start and stop times.

The following example shows you how to create instances that:

  • Start at 9 AM and stop at 5 PM on Monday through Friday
  • Start at 9 AM and stop at 12 PM on Saturday

For this example, you must create two periods. For your own scenario, create the appropriate number of periods.

Using the Instance Scheduler CLI

Connect to the Instance Scheduler CLI, and then run the following commands: 

$ scheduler-cli create-period --stack your_stack_name --region eu-west-1 --name mon-fri-9-5 --begintime 9:00 --endtime 16:59 --weekdays mon-fri
$ scheduler-cli create-period --stack your_stack_name --region eu-west-1 --name sat-9-12 --begintime 9:00 --endtime 11:59 --weekdays sat

Note: Replace your_stack_name with the stack name that you chose in step 4 and eu-west-1 with your own Region.

Using the DynamoDB console

  1. Open the DynamoDB console.
  2. Choose Tables , and then choose the configuration table.
    Note: The Instance Scheduler template automatically creates two DynamoDB tables: state and configuration. The state table stores the state of instances that the template stops and starts. The configuration table allows you to specify the periods and schedules for your requirements.
  3. Choose Explore Table Items .
  4. Choose Create Item .
  5. Choose the JSON view, and then use the following javascript object notation (JSON) template: 
{
  "type": {
    "S": "period"
  },
  "name": {
    "S": "mon-fri-9-5"
  },
  "begintime": {
    "S": "9:00"
  },
  "endtime": {
    "S": "16:59"
  },
  "weekdays": {
    "SS": [
      "mon-fri"
    ]
  }
}

Note: The preceding JSON template creates the first period. Use a similar JSON template for the second period. Make sure to edit the templates for your requirements.

Create a schedule

To create a schedule, you can use the Instance Scheduler CLI, DynamoDB console, or Custom resources.

Using the Instance Scheduler CLI

Run the following command: 

$ scheduler-cli create-schedule --stack your_stack_name --name m-f9-5-sat9-12 --region eu-west-1 --periods mon-fri-9-5,sat-9-12 --timezone UTC

Using the DynamoDB console

  1. Open the DynamoDB console.
  2. Choose Tables , and then choose the configuration table.
  3. Choose Explore Table Items .
  4. Choose Create Item .
  5. Choose the JSON view, and then use the following JSON template: 
{
  "type": {
    "S": "schedule"
  },
  "name": {
    "S": "m-f9-5-sat9-12"
  },
  "timezone": {
    "S": "UTC"
  },
  "periods": {
    "SS": [
      "mon-fri-9-5"
    ]
  }
}

Tag the instance and test the schedule

When you use a CloudFormation stack with the Instance Scheduler, you must define the Instance Scheduler TagName parameter. The default value for this parameter is Schedule .

The Instance Scheduler monitors tags on instances. If the instance tag key matches the defined scheduler tag, then the Instance Scheduler applies the schedule that's set for the instance tag value. For example, a tag's key is set to Schedule and the value is set to m-f9-5-sat9-12 . In this example, the instances start at 9 AM and stop at 5 PM on Monday through Friday. The instances also start at 9 AM and stop at 12 PM on Saturday.

Note: Tag keys and values are case sensitive. The Instance Scheduler doesn't stop running instances if they're manually started outside of the running period. The Instance Scheduler also doesn't start an instance if the instance is stopped manually during the running period, unless the schedule is enforced. For more information, see Schedule definitions.

Use predefined schedules

In addition to custom schedules, you can also use any of the predefined schedules from the configuration table. For example, the following steps test the predefined schedule named running :

  1. Open the Amazon EC2 console.
  2. Choose the stopped instances that you want to tag.
  3. Choose the Tags view, and then choose Manage Tags .
  4. Choose Add Tag .
  5. For Key , enter Schedule .
  6. For Value , enter running .
  7. Choose Save .
  8. Refresh the Amazon EC2 console, and then wait for the Lambda function to be initiated.
    Note: When the Lambda function is initiated and runs without errors, then the Instance State displays as running , depending on the schedule that you're testing. In the CloudWatch console, you can check Lambda metrics for invocations and errors.
  9. Open the DynamoDB console.
  10. Choose Tables , and then choose the state table.
  11. Choose the Explore Table Items and confirm that the tagged instance is started.
    Note: The state data is stored in the state table.
    Important: You can be charged additional costs based on the frequency and duration of the Lambda function that you're using. You can also be charged additional costs for the DynamoDB tables or EventBridge rules that you create.

For cross-account scheduling: Launch the remote stack in secondary accounts

To schedule instances in secondary accounts using the Instance Scheduler, deploy the aws-instance-scheduler-remote CloudFormation template. This template creates the role that allows the Instance Scheduler from the primary account to manage instances in the secondary account.

Note: You must provide the role's ARN as a parameter for the Instance Scheduler stack in the primary account. Make sure to create or update your Instance Scheduler stack with the correct parameter.

  1. Open the AWS Management Console of the secondary account and launch the aws-instance-scheduler-remote CloudFormation template. You can also download the template for future use.
    Note: The template is launched in the US East (N. Virginia) Region by default.
  2. In the navigation bar, select the AWS Region where you want to launch your stack with the template, and then choose Next .
  3. On the Select Template page, verify that you selected the correct template, and then choose Next .
  4. On the Specify Details page, assign a name to your remote stack.
  5. Under Parameters , review and modify the Primary account parameter. Put in the account number of the primary account.
  6. Choose Next .
  7. On the Options page, choose Next .
  8. Review your settings, and then select I acknowledge that AWS CloudFormation might create IAM resources .
  9. Choose Create .
  10. Choose the stack Outputs tab, and then copy the CrossAccountRole value.
  11. From the primary account, choose your CloudFormation stack, and then choose Update .
  12. On the Update stack page, choose Use current template .
  13. In the Cross-account roles parameter, paste the CrossAccountRole value.
  14. Choose Next , and then select I acknowledge that AWS CloudFormation might create IAM resources .
  15. Choose Update Stack .

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support
Read article
Create an IAM policy to control access to EC2 resources using tags

How do I create an IAM policy to control access to Amazon EC2 resources using tags?

Last updated: 2021-09-27

How do I create an AWS Identity and Access Management (IAM) policy that controls access to Amazon Elastic Compute Cloud (Amazon EC2) instances using tags?

Short description

You can control access to smaller deployments of Amazon EC2 instances as follows:

  1. Add a specific tag to the instances you want to grant the users or groups access to.
  2. Create an IAM policy that grants access to any instances with the specific tag.
  3. Attach the IAM policy to the users or groups that you want to access the instances.

Resolution

Add a tag to your group of EC2 instances

Open the Amazon EC2 console, and then add tags to the group of EC2 instances that you want the users or groups to be able to access. If you don't already have a tag, create a new tag.

Note: Be sure to read and understand the tag restrictions before tagging your resources. Amazon EC2 tags are case-sensitive.

Create an IAM policy that grants access to instances with the specific tag

Create an IAM policy that does the following:

  • Allows control over the instances with the tag.
  • Contains a conditional statement that allows access to Amazon EC2 resources if the value of the condition key ec2:ResourceTag/UserName matches the policy variable aws:username . The policy variable ${aws:username} is replaced with the friendly name of the current IAM user when the policy is evaluated by IAM.
  • Allows access to the ec2:Describe* actions for Amazon EC2 resources.
  • Explicitly denies access to the ec2:CreateTags and ec2:DeleteTags actions to prevent users from creating or deleting tags.
    Note: This prevents the user from taking control of an EC2 instance by adding the specific tag to it.

The finished policy looks similar to the following: 

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": "ec2:*",
      "Resource": "*",
      "Condition": {
        "StringEquals": {
          "ec2:ResourceTag/UserName": "${aws:username}"
        }
      }
    },
    {
      "Effect": "Allow",
      "Action": "ec2:Describe*",
      "Resource": "*"
    },
    {
      "Effect": "Deny",
      "Action": [
        "ec2:CreateTags",
        "ec2:DeleteTags"
      ],
      "Resource": "*"
    }
  ]
}

Note: This policy applies to Amazon EC2 instances that use the ec2:ResourceTag condition key. To restrict launching new Amazon EC2 instances using tags, see How can I use IAM policy tags to restrict how an EC2 instance or EBS volume can be created?

Attach the IAM policy to the users or groups you want to access the instances

Finally, attach the IAM policy that you created to the users or groups you want to access the instances. You can attach the IAM policy using the AWS Management Console, AWS CLI, or AWS API.

Granting IAM users required permissions for Amazon EC2 resources

IAM policies for Amazon EC2

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support
Read article
View AWS Activate promotional credits

I received an email with my AWS Activate Founders or Portfolio package information. Where do I find my AWS promotional credit?

Last updated: 2022-03-04

I received an email with my AWS Activate Founders or Portfolio package information. Where do I find my AWS promotional credit?

Resolution

If you receive an email welcoming you to AWS Activate along with benefit information, your AWS Activate Founders or Portfolio package application is approved and processed. Your AWS promotional credits are directly added to the AWS account that you specified on your application.

Check the Credits page of the Billing and Cost Management console to see your account's active credits and promotions.

Getting started with AWS Activate

AWS Activate FAQ

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support
Read article
Resolve issues with an AWS Business support charge for an AWS Activate portfolio package

Why was I charged for AWS Business Support when I have an AWS Activate Portfolio package?

Last updated: 2022-03-04

I was charged for my AWS Support plan, even though I signed up for an AWS Activate Portfolio package that includes a credit for AWS Business Support. How can I resolve this issue?

Resolution

If you had an AWS Support subscription other than a Business-level Support plan before you were approved for an AWS Activate Portfolio package, see AWS Premium Support FAQs, and follow the instructions in Q: How do I cancel my AWS Support subscription? to cancel your support subscription. Then, follow the instructions in the welcome email you received from the AWS Activate team.

If you can't locate the email from the AWS Activate team, or if you have questions about the AWS Activate program, then contact the AWS Activate team at AWS Activate Contact Us.

AWS Activate

AWS Support

Did this article help?

Submit feedback

Do you need billing or technical support?

Contact AWS Support
Read article

Still Thinking?
Give us a try!

We embrace agility in everything we do.
Our onboarding process is both simple and meaningful.
We can't wait to welcome you on AiDOOS!