Clients as we speak could battle to implement correct entry controls and auditing on the person degree when a number of functions are concerned in knowledge entry workflows. The important thing problem is to implement correct least-privilege entry controls primarily based on person id when one software accesses knowledge on behalf of the person in one other software. It forces you to both give all customers broad entry by means of the applying with no auditing, or attempt to implement advanced bespoke options to map roles to customers.
Utilizing AWS IAM Id Heart, now you can propagate person id to a set of AWS providers and decrease the necessity to construct and keep advanced customized methods to vend roles between functions. IAM Id Heart additionally supplies a consolidated view of customers and teams in a single place that the interconnected functions can use for authorization and auditing.
IAM Id Heart permits centralized administration of person entry to AWS accounts and functions utilizing id suppliers (IDPs) like Okta. This enables customers to log in a single time with their present company credentials and seamlessly entry downstream AWS providers supporting id propagation. With IAM Id Heart, Okta person identities and teams might be mechanically synced utilizing SCIM 2.0 for correct person info in AWS.
Amazon EMR Studio is a unified knowledge evaluation setting the place you possibly can develop knowledge engineering and knowledge science functions. Now you can develop and run interactive queries on Amazon Athena from EMR Studio (for extra particulars, confer with Amazon EMR Studio provides interactive question editor powered by Amazon Athena ). Athena customers can entry EMR Studio with out logging in to the AWS Administration Console by enabling federated entry out of your IdP by way of IAM Id Heart. This removes the complexity of sustaining totally different identities and mapping person roles throughout your IdP, EMR Studio, and Athena.
You’ll be able to govern Athena workgroups primarily based on person attributes from Okta to regulate question entry and prices. AWS Lake Formation may use Okta identities to implement fine-grained entry controls by means of granting and revoking permissions.
IAM Id Heart and Okta single sign-on (SSO) integration streamlines entry to EMR Studio and Athena with centralized authentication. Customers can have a well-recognized sign-in expertise with their workforce credentials to securely run queries in Athena. Entry insurance policies on Athena workgroups and Lake Formation permissions present governance primarily based on Okta person profiles.
This weblog publish explains allow single sign-on to EMR Studio utilizing IAM Id Heart integration with Okta. It exhibits propagate Okta identities to Athena and Lake Formation to supply granular entry controls on queries and knowledge. The answer streamlines entry to analytics instruments with centralized authentication utilizing workforce credentials. It leverages AWS IAM Id Heart, Amazon EMR Studio, Amazon Athena, and AWS Lake Formation.
Answer overview
IAM Id Heart permits customers to connect with EMR Studio while not having directors to manually configure AWS Id and Entry Administration (IAM) roles and permissions. It permits mapping of IAM Id Heart teams to present company id roles and teams. Admins can then assign privileges to roles and teams and assign customers to them, enabling granular management over person entry. IAM Id Heart supplies a central repository of all customers in AWS. You’ll be able to create customers and teams straight in IAM Id Heart or join present customers and teams from suppliers like Okta, Ping Id, or Azure AD. It handles authentication by means of your chosen id supply and maintains a person and group listing for EMR Studio entry. Identified person identities and logged knowledge entry facilitates compliance by means of auditing person entry in AWS CloudTrail.
The next diagram illustrates the answer structure.
The EMR Studio workflow consists of the next high-level steps:
- The tip-user launches EMR Studio utilizing the AWS entry portal URL. This URL is supplied by an IAM Id Heart administrator by way of the IAM Id Heart dashboard.
- The URL redirects the end-user to the workforce IdP Okta, the place the person enters workforce id credentials.
- After profitable authentication, the person will likely be logged in to the AWS console as a federated person.
- The person opens EMR Studio and navigates to the Athena question editor utilizing the hyperlink obtainable on EMR Studio.
- The person selects the right workgroup as per the person function to run Athena queries.
- The question outcomes are saved in separate Amazon Easy Storage Service (Amazon S3) places with a prefix that’s primarily based on person id.
To implement the answer, we full the next steps:
- Combine Okta with IAM Id Heart to sync customers and teams.
- Combine IAM Id Heart with EMR Studio.
- Assign customers or teams from IAM Id Heart to EMR Studio.
- Arrange Lake Formation with IAM Id Heart.
- Configure granular role-based entitlements utilizing Lake Formation on propagated company identities.
- Arrange workgroups in Athena for governing entry.
- Arrange Amazon S3 entry grants for fine-grained entry to Amazon S3 sources like buckets, prefixes, or objects.
- Entry EMR Studio by means of the AWS entry portal utilizing IAM Id Heart.
- Run queries on the Athena SQL editor in EMR Studio.
- Evaluate the end-to-end audit path of workforce id.
Conditions
To comply with alongside this publish, you must have the next:
- An AWS account – When you don’t have one, you possibly can enroll right here.
- An Okta account that has an energetic subscription – You want an administrator function to arrange the applying on Okta. When you’re new to Okta, you possibly can join a free trial or a developer account.
For directions to configure Okta with IAM Id Heart, confer with Configure SAML and SCIM with Okta and IAM Id Heart.
Combine Okta with IAM Id Heart to sync customers and teams
After you may have efficiently synced customers or teams from Okta to IAM Id Heart, you possibly can see them on the IAM Id Heart console, as proven within the following screenshot. For this publish, we created and synced two person teams:
- Information Engineer
- Information Scientists
Subsequent, create a trusted token issuer in IAM Id Heart:
- On the IAM Id Heart console, select Settings within the navigation pane.
- Select Create trusted token issuer.
- For Issuer URL, enter the URL of the trusted token issuer.
- For Trusted token issuer identify, enter Okta.
- For Map attributes¸ map the IdP attribute E mail to the IAM Id Heart attribute E mail.
- Select Create trusted token issuer.
The next screenshot exhibits your new trusted token issuer on the IAM Id Heart console.
Combine IAM Id Heart with EMR Studio
We begin with making a trusted id propagation enabled in EMR Studio.
An EMR Studio administrator should carry out the steps to configure EMR Studio as an IAM Id Heart-enabled software. This permits EMR Studio to find and hook up with IAM Id Heart mechanically to obtain sign-in and person listing providers.
The purpose of enabling EMR Studio as an IAM Id Heart-managed software is so you possibly can management person and group permissions from inside IAM Id Heart or from a supply third-party IdP that’s built-in with it (Okta on this case). When your customers check in to EMR Studio, for instance data-engineer or data-scientist, it checks their teams in IAM Id Heart, and these are mapped to roles and entitlements in Lake Formation. On this method, a gaggle can map to a Lake Formation database function that permits learn entry to a set of tables or columns.
The next steps present create EMR Studio as an AWS-managed software with IAM Id Heart, then we see how the downstream functions like Lake Formation and Athena propagate these roles and entitlements utilizing present company credentials.
- On the Amazon EMR console, navigate to EMR Studio.
- Select Create a Studio.
- For Setup choices, choose Customized.
- For Studio identify, enter a reputation.
- For S3 location for Workspace storage, choose Choose present location and enter the Amazon S3 location.
6. Configure permission particulars for the EMR Studio.
Observe that if you select View permission particulars below Service function, a brand new pop-up window will open. You should create an IAM function with the identical insurance policies as proven within the pop-up window. You need to use the identical on your service function and IAM function.
- On the Create a Studio web page, for Authentication, choose AWS IAM Id Heart.
- For Consumer function, select your person function.
- Beneath Trusted id propagation, choose Allow trusted id propagation.
- Beneath Utility entry, choose Solely assigned customers and teams.
- For VPC, enter your VPC.
- For Subnets, enter your subnet.
- For Safety and entry, choose Default safety group.
- Select Create Studio.
You must now see an IAM Id Heart-enabled EMR Studio on the Amazon EMR console.
After the EMR Studio administrator finishes creating the trusted id propagation-enabled EMR Studio and saves the configuration, the occasion of the EMR Studio seems as an IAM Id Heart-enabled software on the IAM Id Heart console.
Assign customers or teams from IAM Id Heart to EMR Studio
You’ll be able to assign customers and teams out of your IAM Id Heart listing to the EMR Studio software after syncing with IAM. The EMR Studio administrator decides which IAM Id Heart customers or teams to incorporate within the app. For instance, you probably have 10 whole teams in IAM Id Heart however don’t need all of them accessing this occasion of EMR Studio, you possibly can choose which teams to incorporate within the EMR Studio-enabled IAM app.
The next steps assign teams to EMR Studio-enabled IAM Id Heart software:
- On the EMR Studio console, navigate to the brand new EMR Studio occasion.
- On the Assigned teams tab, select Assign teams.
- Select which IAM Id Heart teams you need to embody within the software. For instance, chances are you’ll select the Information-Scientist and Information-Engineer teams.
- Select Executed.
This enables the EMR Studio administrator to decide on particular IAM Id Heart teams to be assigned entry to this particular occasion built-in with IAM Id Heart. Solely the chosen teams will likely be synced and given entry, not all teams from the IAM Id Heart listing.
Arrange Lake Formation with IAM Id Heart
To arrange Lake Formation with IAM Id Heart, just remember to have configured Okta because the IdP for IAM Id Heart, and ensure that the customers and teams type Okta are actually obtainable in IAM Id Heart. Then full the next steps:
- On the Lake Formation console, select IAM Id Heart Integration below Administration within the navigation pane.
You will note the message “IAM Id Heart enabled” together with the ARN for the IAM Id Heart software.
- Select Create.
In a couple of minutes, you will note a message indicating that Lake Formation has been efficiently built-in along with your centralized IAM identities from Okta Id Heart. Particularly, the message will state “Efficiently created id heart integration with software ARN,” signifying the mixing is now in place between Lake Formation and the identities managed in Okta.
Configure granular role-based entitlements utilizing Lake Formation on propagated company identities
We’ll now arrange granular entitlements for our knowledge entry in Lake Formation. For this publish, we summarize the steps wanted to make use of the prevailing company identities on the Lake Formation console to supply related controls and governance on the information, which we’ll later question by means of the Athena question editor. To find out about organising databases and tables in Lake Formation, confer with Getting began with AWS Lake Formation
This publish is not going to go into the total particulars about Lake Formation. As a substitute, we’ll deal with a brand new functionality that has been launched in Lake Formation—the flexibility to arrange permissions primarily based in your present company identities which are synchronized with IAM Id Heart.
This integration permits Lake Formation to make use of your group’s IdP and entry administration insurance policies to regulate permissions to knowledge lakes. Somewhat than defining permissions from scratch particularly for Lake Formation, now you can depend on your present customers, teams, and entry controls to find out who can entry knowledge catalogs and underlying knowledge sources. Total, this new integration with IAM Id Heart makes it simple to handle permissions on your knowledge lake workloads utilizing your company identities. It reduces the executive overhead of protecting permissions aligned throughout separate methods. As AWS continues enhancing Lake Formation, options like it will additional enhance its viability as a full-featured knowledge lake administration setting.
On this publish, we created a database referred to as zipcode-db-tip and granted full entry to the person group Information-Engineer to question on the underlying desk within the database. Full the next steps:
- On the Lake Formation console, select Grant knowledge lake permissions.
- For Principals, choose IAM Id Heart.
- For Customers and teams, choose Information-Engineer.
- For LF-Tags or catalog sources, choose Named Information Catalog sources.
- For Databases, select
zipcode-db-tip. - For Tables, select
tip-zipcode.
Equally, we have to present the related entry on the underlying tables to the customers and teams for them to have the ability to question on the information.
- Repeat the previous steps to supply entry to the Information-Engineer group to have the ability to question on the information.
- For Desk permissions, choose Choose, Describe, and Tremendous.
- For Information permissions, choose All knowledge entry.
You’ll be able to grant selective entry on rows and feedback as per your particular necessities.
Arrange workgroups in Athena
Athena workgroups are an AWS characteristic that permits you to isolate knowledge and queries inside an AWS account. It supplies a solution to segregate knowledge and management entry so that every group can solely entry the information that’s related to them. Athena workgroups are helpful for organizations that need to limit entry to delicate datasets or assist stop queries from impacting one another. Once you create a workgroup, you possibly can assign customers and roles to it. Queries launched inside a workgroup will run with the entry controls and settings configured for that workgroup. They permit governance, safety, and useful resource controls at a granular degree. Athena workgroups are an essential characteristic for managing and optimizing Athena utilization throughout giant organizations.
On this publish, we create a workgroup particularly for members of our Information Engineering workforce. Later, when logged in below Information Engineer person profiles, we run queries from inside this workgroup to exhibit how entry to Athena workgroups might be restricted primarily based on the person profile. This enables governance insurance policies to be enforced, ensuring customers can solely entry permitted datasets and queries primarily based on their function.
- On the Athena console, select Workgroups below Administration within the navigation pane.
- Select Create workgroup.
- For Authentication, choose AWS Id Heart.
- For Service function to authorize Athena, choose Create and use a brand new service function.
- For Service function identify, enter a reputation on your function.
- For Location of question outcome, enter an Amazon S3 location for saving your Athena question outcomes.
This can be a necessary subject if you specify IAM Id Heart for authentication.
After you create the workgroup, you must assign customers and teams to it. For this publish, we create a workgroup named data-engineer and assign the group Information-Engineer (propagated by means of the trusted id propagation from IAM Id Heart).
- On the Teams tab on the data-engineer particulars web page, choose the person group to assign and select Assign teams.
Arrange Amazon S3 entry grants to separate the question outcomes for every workforce id
Subsequent, we arrange Amazon S3 grants.
You’ll be able to watch the next video to arrange the grants or confer with Use Amazon EMR with S3 Entry Grants to scale Spark entry Amazon S3 for directions.
Provoke login by means of AWS federated entry utilizing the IAM Id Heart entry portal
Now we’re prepared to connect with EMR Studio and federated login utilizing IAM Id Heart authentication:
- On the IAM Id Heart console, navigate to the dashboard and select the AWS entry portal URL.
- A browser pop-up directs you to the Okta login web page, the place you enter your Okta credentials.
- After profitable authentication, you’ll be logged in to the AWS console as a federated person.
- Select the EMR Studio software.
- After you federate to EMR Studio, select Question Editor within the navigation pane to open a brand new tab with the Athena question editor.
The next video exhibits a federated person utilizing the AWS entry portal URL to entry EMR Studio utilizing IAM Id Heart authentication.
Run queries with granular entry on the editor
On EMR Studio, the person can open the Athena question editor after which specify the right workgroup within the question editor to run the queries.
The information engineer can question solely the tables on which the person has entry. The question outcomes will seem below the S3 prefix, which is separate for every workforce id.
Evaluate the end-to-end audit path of workforce id
The IAM Id Heart administrator can look into the downstream apps which are trusted for id propagation, as proven within the following screenshot of the IAM Id Heart console.
On the CloudTrail console, the occasion historical past shows the occasion identify and useful resource accessed by the particular workforce id.
Once you select an occasion in CloudTrail, the auditors can see the distinctive person ID that accessed the underlying AWS Analytics providers.
Clear up
Full the next steps to scrub up your sources:
- Delete the Okta functions that you just created to combine with IAM Id Heart.
- Delete IAM Id Heart configuration.
- Delete the EMR Studio that you just created for testing.
- Delete the IAM function that you just created for IAM Id Heart and EMR Studio integration.
Conclusion
On this publish, we confirmed you an in depth walkthrough to convey your workforce id to EMR Studio and propagate the id to linked AWS functions like Athena and Lake Formation. This resolution supplies your workforce with a well-recognized sign-in expertise, with out the necessity to keep in mind further credentials or keep advanced function mapping throughout totally different analytics methods. As well as, it supplies auditors with end-to-end visibility into workforce identities and their entry to analytics providers.
To be taught extra about trusted id propagation and EMR Studio, confer with Combine Amazon EMR with AWS IAM Id Heart.
In regards to the authors
Manjit Chakraborty is a Senior Options Architect at AWS. He’s a Seasoned & Outcome pushed skilled with intensive expertise in Monetary area having labored with clients on advising, designing, main, and implementing core-business enterprise options throughout the globe. In his spare time, Manjit enjoys fishing, practising martial arts and enjoying along with his daughter.
Neeraj Roy is a Principal Options Architect at AWS primarily based out of London. He works with International Monetary Companies clients to speed up their AWS journey. In his spare time, he enjoys studying and spending time along with his household.