Note: A version of this article was originally posted on DZone. What you are reading now has been enhanced with more details around the suggested solution.
Keeping with the above example, next is the border control agent asking for your passport. For computer systems, the equivalent is authentication, which, like a physical passport, proves that you are who you claim to be. Here you will find services such as Kerberos, or token-based systems such as JWT or OpenID.
AWS IAM in a Nutshell
- Create AWS IAM roles that have permission to access the S3 service endpoints (defined as a set of actions) and
- define S3 bucket policies to allow or deny access to the contained objects.
Requirements of Access Control in Data Lakes
- IAM Integration – Since IAM bucket policies are single JSON structures containing all access roles for a given bucket, they are very error-prone. There is a need for a better, easy-to-use access control system to manage granular access control. Any new system must integrate with the existing IAM service.
- Distributed Access – The system should let you delegate permissions easily to data owners so that they can control access to their data, removing the need for time-consuming, cross-functional processes.
- Single Unified Layer – Ideally this access control mechanism should act as a single unified security layer, covering not only authentication and authorization but also audit event logging.
- Unified Governance – No access control system is complete without the capability to see who is using what resources in your data lake.
- Privacy Compliance – Global data privacy regulations demand that data be discovered automatically, tagged correctly, and that the right information is always presented to the right people without making numerous copies of the data.
- Sensitive Data Protection – Besides making the right information available to the right roles, sensitive data must also be obfuscated for protection against threats both internal and external.
IAM Slowing Down Data Projects?
- have someone capable to write highly complex JSON policies (which can grow to multiple pages, even exceeding the maximum allowed length of up to 20KB),
- ensure all business units are included as either users, groups or roles that should have access,
- assign the policies to the right entities, and
- verify access is working as expected.
Fine-grained Access Control and Okera
- Set up access to storage systemsone time for the Okera services, and
- Delegate business data administration to the lines of business thatowns the data.