BellaDati can import data and files from Amazon S3.


Files metadata are saved to the data set and files can be saved to the BellaDati Media gallery.

Connecting to S3

From Data Source Connectors page select S3.


Connection parameters


Provide your unique Account key

Provide your unique Account secret

Choose AWS region what has your bucket is assigned to

Provide with your Bucket name

You can select time interval Modified from-to. If data were modified in this time interval, they will be used in import

You can also use relative timing with Max file age in minutes old.

Row prefix regex defines according to what format, should rows be downloaded. For example "\.json$" will search for all files that ends with .json

Column file patterns defines according to what format, should columns be downloaded. Each column regex should be on new line. For example ^A.*\.json$ and ^B.*\.json$. This will download jsons starting with A on first column, and starting with B on second column.

S3 Functions

  • String getS3Text() - Gets file
  • String getS3Text(int columnIndex) - Gets file from selected column
  • JSONElement getS3JSON() - Gets json file
  • JSONElement getS3JSON(int columnIndex) - Gets json file from selected column
  • String saveS3MediaFile() - Gets picture and stores it to dataset, returns mediaId
  • String saveS3MediaFile(boolean skipIfExists) - Gets picture and stores it to media gallery, returns mediaId. With true parameter can skip already downloaded file
  • String saveS3MediaFile(int columnIndex, boolean skipIfExists) - Gets picture from selected column and stores it to media gallery, returns mediaId. With true parameter can skip already downloaded file
  • connectS3(String accountKey, String accountSecret, String region, String bucketName, Closure closure) - connects to S3 and sets the "client" as property to the closure
  • iterateS3Objects(S3Client client, String bucketName, Closure closure) - iterates objects in S3 and sets "client" and "object" as properties to the closure
  • deleteS3Object(S3Client client, String bucketName, String key)
  • deleteS3Object(S3Client client, String bucketName, S3Object object)

Example - Load data from JSON file stored in S3

Data returned from S3

S3 connector loads the list of JSON files based on the connection parameters

Getting data from loaded JSON files

In order to get the new data from the related JSON files, add a new column to the Import settings. And define the following transformation script:

getS3JSON(1)

This script will load JSON data from the related file.

Loading data from selected JSON element

For saving the data from the loaded JSON element, there should be created a new column in the import settings. The element value will be loaded by using the following transformation script (this script will load the value from JSON element "camera_id"):

getS3JSON(1).get("camera_id")

Set up Read only policy

This configuration is used 


You can set permissions for your for whole S3 or even for specific bucket using IAM.

  1. Create your bucket with default settings
  2. Get your ARN, this can be found under Properties of your bucket

  3. Create new Policy under IAM. You can use this dummy policy. In Resource instead of ARN_CODE add your ARN

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "s3:List*",
                "s3:Describe*",
                "s3-object-lambda:List*"
            ],
            "Resource": "*"
        },
		{
			"Effect": "Allow",
            "Action": [
                "s3:Get*",
                "s3:List*",
                "s3:Describe*",
                "s3-object-lambda:Get*",
                "s3-object-lambda:List*"
            ],
            "Resource": "ARN_CODE/*"
		}
    ]
}
  4. Go Next and Next
  5. Enter name of your policy and Save it
  6. Go to Users and search for your user
  7. Add permissions
  8. Select "Attach existing policies directly"
  9. Search for name of policy set in step 5
  10. Select it and click "Next: review" then "Add permissions"


You can specify roles on bucket level

  1. Use steps from previous IAM guide.

  2. Create new policy, that looks like this and assign it to user

    {
    "Version”: “2012-10-17",
    "Statement": [
        {
            "Effect": "Allow”,
            "Action": "s3:ListAllMyBuckets",
            "Resource”: "*"
        }
    ]
}
  3. Create new policy in your bucket. Go to buckets, open your bucket and go to Permissions

  4. Edit bucket policy and use this dummy JSON

    {
    "Version": "2012-10-17",
    "Statement": [
        {
            "Sid": "Statement1",
            "Effect": "Allow",
            "Principal": {
                "AWS": "USER_ARN_CODE"
            },
            "Action": [
                "s3:GetObject",
                "s3:ListBucket"
            ],
            "Resource": [
                "BUCKET_ARN_CODE/*",
                "BUCKET_ARN_CODE"
            ]
        }
    ]
}
    USER_ARN_CODE, can be found in user sumary. It can look like arn:aws:iam::512345678907:user/nameOfYourUser
    BUCKET_ARN_CODE, can be found in bucket properties. It can look like arn:aws:s3:::bucketName
  • No labels