appRules
  • Welcome to appRules®
  • PLATFORM
    • appRules® Concepts & Features
    • Platform Overview
    • Platform Features
      • Sourced Values
        • Overview
        • Sourced Values List
      • Actions
        • Overview
        • Action Categories
          • Actions-DataSource
            • Accounting
            • AIMachineLearning
            • Analytics
            • ApplicationPlatforms
            • BigData
            • CalendarsAndContactMgrs
            • CloudStorage
            • Collaboration
            • CRM
            • CustomerService
            • Databases
            • DataWarehouse
            • ECommercePlatforms
            • EmailSystems
            • ERP
            • ExpenseManagement
            • Files
            • Financial
            • Marketing
            • Miscelleanous
            • NoSQL
            • PaymentProcessing
            • PLM
            • ProjectManagement
            • Search
            • Services
            • ShippingAndMailing
            • Social
            • Spreadsheets
          • Actions-Execution
          • Actions-Fields-Field
          • Actions-Fields-Library-Financial
          • Actions-Fields-Library-Math
          • Actions-Utility-Automation
          • Actions-Utility-FileSystem
      • Conditions
        • Overview
        • Condition Categories
          • Conditions-Cache
          • Conditions-Custom
          • Conditions-Fields
          • Conditions-FileSystem
          • Conditions-Records
          • Conditions-Results
          • Conditions-Workflow
      • Workflow Activities
        • Control Flow Activities
        • Base Activities
        • Dialog Activities
        • Base Data Activities
      • System Settings
        • Runtime Settings
        • Users
        • Data Sources
          • Overview
          • Configuring Connection Settings
          • Generating Metadata
          • Connection Settings
            • Accounting
              • Exact Online
              • Raisers Edge NXT
              • Xero
              • Quickbooks
              • Quickbooks Online
              • Quickbooks POS
              • Freshbooks
              • Sage Business Cloud Accounting
              • Sage Intacct
              • Sage50 UK
              • Reckon
            • Application Platforms
              • Alfresco Platform
              • Kintone Platform
              • Quickbase Platform
              • Veeva Vault
            • AI & Machine Learning
              • DataRobot
              • Salesforce Einstein
              • Splunk
            • Analytics
              • Google Analytics
              • Adobe Analytics
                • Adobe Custom oAuth App
              • Azure Analysis Services
              • SAS
              • Youtube Analytics
            • Big Data
              • Apache HBase
              • Apache Hive
              • Apache HDFS
              • Apache Spark
              • Greenplum
              • HPCC LexisNexis
            • Calendar & Contact Managers
              • Google Calendar
              • Google Contacts
            • Cloud Storage
              • Azure Table Storage
              • Amazon S3
              • IBM Cloud Object Storage
              • Wasabi
            • Collaboration
              • Airtable
              • Asana
              • Box
              • Dropbox
              • Exchange
              • Google Drive
              • Microsoft Teams
              • Office365
              • Microsoft OneDrive
              • Microsoft OneNote
              • Salesforce Chatter
              • SharePoint
              • Slack
              • Smartsheet
            • CRM
              • Act! CRM
              • Bullhorn CRM
              • Microsoft Dynamics 365 Sales
              • Microsoft Dynamics CRM
              • Highrise
              • Oracle Sales Cloud
              • Salesforce
              • SAP Hybris C4C
              • Sugar CRM
              • Suite CRM
              • Veeva CRM
              • Zoho CRM
            • Customer Service & Support
              • ServiceNow
              • Zendesk
              • Jira
            • Data Warehouse
              • Azure Data Lake Storage Gen1
              • Azure Data Lake Storage Gen2
              • Google BigQuery
              • Redshift
              • Databricks
              • Snowflake
              • Azure Synapse
            • E-Commerce Platforms
              • Amazon MarketPlace
              • Magento
              • Shopify
              • Woocommerce
            • Electronic Signature
              • Docusign
            • Email
              • Gmail
              • Email
            • ERP
              • NetSuite
              • Odoo
              • Acumatica
              • Dynamics NAV
              • Dynamics GP
              • Microsoft Dynamics 365 FinOp
              • Microsoft Dynamics365 Business Central
                • Business Central Endpoints
              • SAP Business One
              • SAP ERP
            • Expense Management
              • SAP Concur
            • Files
              • CSV Database
              • Fixed Length File
              • Delimited File
            • Marketing
              • Bing Ads
              • Marketo
              • MailChimp
              • HubSpot
              • Eloqua
              • Salesforce Marketing Cloud
            • NoSQL
              • Cassandra
                • Cassandra Advanced Settings
              • Azure CosmosDB
              • DynamoDB
              • Couchbase
              • MongoDB
              • IBM Cloudant
              • Redis
            • Payment Processing
              • Authorize.Net
              • Square
              • Paypal
              • Stripe
            • Product Lifecycle Management (PLM)
              • Propel
            • Relational Databases
              • appRules Embedded Database
              • Firebird
              • IBM DB2
              • IBM Informix
              • Microsoft Access
              • MySQL
              • MariaDB
              • Oracle
              • PostgreSQL
              • Microsoft SQL Server
              • Microsoft SQL Server CE
              • SQL Azure
              • SQLlite
              • Vertica
            • Search
              • ElasticSearch
              • Bing
              • Google Search
            • Shipping and Mailing
              • Fedex
              • UPS
              • USPS
            • Social Networks
              • Linkedin
              • Facebook
              • Instagram
              • Twitter
            • Services
              • REST
              • RSS
              • XML
                • Connecting to XML Datasources
                • Parsing Hierarchical Data
                • Raw Data
              • Active Directory
              • Amazon Athena
              • Microsoft Dataverse
              • JSON
              • LDAP
              • GraphQL
            • Spreadsheets
              • Excel
              • Excel Services
              • Excel Online
              • GoogleSheets
          • SQL Compliance
            • Accounting
              • Exact Online
              • Raisers Edge NXT
              • Xero
              • Quickbooks
              • Quickbooks Online
              • QuickBooks POS
              • Freshbooks
              • Reckon
              • Sage Intacct
              • Sage Business Cloud Accounting
              • Sage50 UK
            • AI & Machine Learning
              • Salesforce Einstein
              • DataRobot
              • Splunk
            • Analytics
              • Google Analytics
              • Youtube Analytics
              • Azure Analysis Services
              • Adobe Analytics
              • SAS
            • Application Platforms
              • Alfresco Platform
              • Kintone Platform
              • Quick Base Platform
              • Veeva Vault
            • Big Data
              • Apache HBase
              • Apache HDFS
              • Apache Hive
              • Apache Spark
              • Greenplum
              • HPCC Systems
            • Calendar & Contact Managers
              • Google Calendar
              • Google Contacts
            • Cloud Storage
              • Azure Table Storage
              • Amazon S3
              • Wasabi
              • IBM Cloud Object Storage
            • Collaboration
              • Airtable
              • Asana
              • Box
              • Dropbox
              • Google Drive
              • Microsoft Teams
              • Microsoft OneDrive
              • Microsoft OneNote
              • Microsoft Exchange
              • Office365
              • Salesforce Chatter
              • SharePoint
              • Slack
              • Smartsheet
            • CRM
              • Act! CRM
              • Bullhorn CRM
              • Highrise
              • Microsoft Dynamics CRM
              • Microsoft Dynamics 365 Sales
              • Oracle Sales Cloud
              • Salesforce
              • SAP Hybris C4C
              • Sugar CRM
              • Veeva CRM
              • Zoho CRM
            • Customer Service & Support
              • ServiceNow
              • Zendesk
              • Jira
            • E-commerce Platforms
              • Woocommerce
              • Magento
              • Shopify
              • Amazon Marketplace
            • DataWarehouse
              • Azure Data Lake Storage Gen1
              • Azure Data Lake Storage Gen2
              • Azure Synapse
              • Databricks
              • Google BigQuery
              • Redshift
              • Snowflake
            • Email
              • Gmail
            • ERP
              • Acumatica
              • NetSuiteSQL
              • NetSuiteTalk
              • Odoo
              • Dynamics NAV
              • Dynamics GP
              • Microsoft Dynamics 365 FinOp
              • Microsoft Dynamics365 Business Central
              • SAP Business One
              • SAP NetWeaver
              • Workday
            • Expense Management
              • SAP Concur
            • Files
              • CSV Database
            • Marketing
              • Bing Ads
              • Google Ads
              • HubSpot
              • MailChimp
              • Marketo
              • Oracle Eloqua
              • Salesforce Marketing Cloud
              • Survey Monkey
              • Youtube 2020
            • NoSQL
              • Cassandra
              • Azure CosmosDB
              • Amazon SimpleDB
              • Amazon DynamoDB
              • Couchbase
              • MongoDB
              • IBM Cloudant
              • Redis
            • Payment Processing
              • Authorize.Net
              • Paypal
              • Stripe
              • Square
            • Product Lifecycle Management (PLM)
              • Propel
            • Project Management
              • Basecamp
              • Microsoft Project
            • Search
              • ElasticSearch
              • Bing
              • Google Search
            • Shipping and Mailing
              • FedEx
              • UPS
              • USPS
            • Spreadsheets
              • Excel Online
              • Excel
              • Excel Services
              • GoogleSheets
            • Social Networks
              • Facebook
              • Instagram
              • Linkedin
              • Twitter
            • Services
              • Active Directory
              • Amazon Athena
              • JSON
              • LDAP
              • Microsoft Dataverse
              • OData
              • REST
              • RSS
              • XML
        • Projects
          • Overview
          • Project Settings
      • Utilities
        • DataSource Browser
        • Job Monitor
        • Logs/Statistics
        • CheckIn/CheckOut
        • Master Admin Options
      • Creating & Running Projects
        • Creating a New Project
        • Orchestrating Workflows with Designer
          • Designer Home Page Overview
          • Using the appRules Designer
        • Reading Data
        • Mapping Data
        • Writing Data
        • Project Validation
        • Running Projects
          • Running from the Home Page
          • Using the Web API
          • Using the Scheduler
        • Monitoring Jobs
        • Logging
      • Flow Data Migration
        • Overview
        • Basic Flow Data Migrations
        • Bulk Loader Flow Data Migrations
    • Self-Hosting
      • Software Installation
        • Installation Prerequisites
        • Running the Installer
        • HostService Settings
        • IIS Settings
        • Web API Settings
        • Scheduler
        • Creating a New Project Database
      • Application Configuration
      • appRules Embedded Database DBA Tool
  • PLATFORM PRODUCTS
    • appRules Data Migration Engine
      • Overview
      • Flow Data Migration
      • The Migrate Data Activity
        • Overview
        • Initializing the Migrate Data Activity
        • Target Entities Collection
          • Overview
          • Toolbar
          • Target Entity Properties
        • Basic Data Migration
        • Matching Schema Migration
        • One-To-Many Migration
        • Entities to Csv Migration
        • Data Replication
        • Data Synchronization
        • Bidirectional Data Synchronization
      • Field Mapping & Transformation
      • Reviewing the Project Settings
      • Validations, Error Handling & Notifications
      • Job Execution & Monitoring
      • Evaluating Data Migration Job Results
    • appRules Integration & Automation Engine
      • Overview
      • Flow Data Migration
      • Process Automation, Business Rules & Decision Support
        • Overview
        • Control Flow Activities
          • If
          • While
        • Conditions & Actions
          • Overview
          • Conditions
          • Actions
        • Evaluating Conditions
        • Performing Actions
        • Decision Tables
        • Executing Rule Sets
    • appRules Enterprise 360
      • Overview
      • Data Migration
      • Integration & Automation
  • Samples
    • Samples Overview
    • Sample Projects
      • Data Migration, Replication & Synchronization
        • Replicate Dynamics365 In Snowflake
        • Migrate Parent/Child Records Using Target / Bulk Loader
        • Export SQL Table to CSV Using Flow
        • One-to-Many Data Migration
        • Batch Data Source Entities to Files
      • Integration & Automation
        • Decision Table - Customer Discount
        • Iterate Deleted Salesforce Records
        • Use CSV Folder As SQL Database
        • Conditional Actions Execution
        • Round Robin Lead Assignment
      • Snippets
        • Snippet - Arguments, Variables, Dynamic Values & Sourced Values
        • Snippet - Counters & Running Totals
        • Snippet - Executing Financial Functions
        • Snippet - Infinite Loop
        • Using Composite Keys
      • Miscellaneous
        • Custom Queries
        • Parent-Child: Child Project
        • Parent-Child: Parent Project
        • Scheduled System Maintenance
        • Target Activity Errors & Logs
        • Task Parallelism
  • APPENDIX
    • Miscellaneous
      • Common Connection Properties
      • Troubleshooting a Connection
      • Dynamics365 - Connection Settings
      • Dynamics365 App Registration
      • Getting the oAuth Access and Refresh Tokens
      • Editing Windows Environment Variables
      • PickLists
      • Creating a Custom Azure oAuth App
      • Using Kerberos
      • Stored Procedures
    • FAQs
Powered by GitBook
On this page
  • Connecting to Amazon S3
  • Authenticating to Amazon S3
Export as PDF
  1. PLATFORM
  2. Platform Features
  3. System Settings
  4. Data Sources
  5. Connection Settings
  6. Cloud Storage

Amazon S3

Connecting to Amazon S3

Specify the following to connect to data:

  • CustomURL: Specify the base S3 service URL if it has a different URL from 'amazonaws.com'. Make sure to specify the full URL. For example: 'http://127.0.0.1:9000'.

  • AWSRegion: Set this to the region where your Amazon S3 data is hosted.

Authenticating to Amazon S3

There are several authentication methods available for connecting to Amazon S3 including: authenticating with Root Credentials, Temporary Credentials, as an AWS Role (from an EC2 Instance or by specifying the root credentials), using SSO and using a Credential File.

Obtain AWS Keys

To obtain the credentials for an IAM user, follow the steps below:

  1. Sign into the IAM console.

  2. In the navigation pane, select Users.

  3. To create or manage the access keys for a user, select the user and then go to the Security Credentials tab.

To obtain the credentials for your AWS root account, follow the steps below:

  1. Sign into the AWS Management console with the credentials for your root account.

  2. Select your account name or number and select My Security Credentials in the menu that is displayed.

  3. Click Continue to Security Credentials and expand the "Access Keys" section to manage or create root account access keys.

Root Credentials

To authenticate using account root credentials, set the following:

  • AuthScheme: Set this to AwsRootKeys.

  • AWSAccessKey: The access key associated with the AWS root account.

  • AWSSecretKey: The secret key associated with the AWS root account.

Note: Use of this authentication scheme is discouraged by Amazon for anything but simple tests. The account root credentials have the full permissions of the user, making this the least secure authentication method.

Temporary Credentials

To authenticate using temporary credentials, specify the following:

  • AuthScheme: Set this to TemporaryCredentials.

  • AWSAccessKey: The access key of the IAM user to assume the role for.

  • AWSSecretKey: The secret key of the IAM user to assume the role for.

The provider can now request resources using the same permissions provided by long-term credentials (such as IAM user credentials) for the lifespan of the temporary credentials.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the provider to attempt to retrieve credentials for the specified role.

  • AWSExternalId: Only if required when you assume a role in another account.

EC2 Instances

If you are using the provider from an EC2 Instance and have an IAM Role assigned to the instance, you can use the IAM Role to authenticate. To do so, set the following properties to authenticate:

  • AuthScheme: Set this to AwsEC2Roles.

Do not specify AWSAccessKey and AWSSecretKey because the provider will automatically obtain your IAM Role credentials and authenticate with them.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the provider to attempt to retrieve credentials for the specified role.

  • AWSExternalId: Only if required when you assume a role in another account.

IMDSv2 Support

The Amazon S3 provider now supports IMDSv2. Unlike IMDSv1, the new version requires an authentication token. Endpoints and response are the same in both versions. In IMDSv2, the Amazon S3 provider first attempts to retrieve the IMDSv2 metadata token and then uses it to call AWS metadata endpoints. If it is unable to retrieve the token, the provider reverts to IMDSv1.

AWS IAM Roles

In many situations it may be preferable to use an IAM role for authentication instead of the direct security credentials of an AWS root user.

To authenticate as an AWS role, set the following:

  • AuthScheme: Set this to AwsIAMRoles.

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the provider to attempt to retrieve credentials for the specified role.

  • AWSExternalId: Only if required when you assume a role in another account.

  • AWSAccessKey: The access key of the IAM user to assume the role for.

  • AWSSecretKey: The secret key of the IAM user to assume the role for.

Note: Roles may not be used when specifying the AWSAccessKey and AWSSecretKey of an AWS root user.

ADFS

Set the AuthScheme to ADFS. The following connection properties need to be set:

  • User: Set this to your ADFS username.

  • Password: Set this to your ADFS password.

  • SSOLoginURL: Set this to the login URL used by the SSO provider.

Below is an example connection string:

AuthScheme=ADFS; AWSRegion=Ireland; User=user@cdata.com; Password=CH8WerW121235647iCa6; SSOLoginURL='https://adfs.domain.com'; AWSRoleArn=arn:aws:iam::1234:role/ADFS_SSO; AWSPrincipalArn=arn:aws:iam::1234:saml-provider/ADFSProvider;

ADFS Integrated

To use the ADFS Integrated flow, specify the SSOLoginURL and leave the username and password empty.

Okta

Set the AuthScheme to Okta. The following connection properties are used to authenticate through Okta:

  • User: Set to your Okta user.

  • Password: Set to your Okta password.

  • SSOLoginURL: Set to the login URL used by the SSO provider.

If you are:

  • using a trusted application or proxy that overrides the Okta client request

  • configuring MFA

then you need to use combinations of SSOProperties input parameters to authenticate using Okta. Otherwise, you do not need to set any of these values.

In SSOProperties when required, set these input parameters:

  • APIToken: When authenticating a user via a trusted application or proxy that overrides the Okta client request context, set this to the API Token the customer created from the Okta organization.

  • MFAType: Set this if you have configured the MFA flow. Currently we support the following types: OktaVerify, Email, and SMS.

  • MFAPassCode: Set this only if you have configured the MFA flow. If you set this to empty or an invalid value, the provider issues a one-time password challenge to your device or email. After the passcode is received, reopen the connection where the retrieved one-time password value is set to the MFAPassCode connection property.

  • MFARememberDevice: Okta supports remembering devices when MFA is required. If remembering devices is allowed according to the configured authentication policies, the provider sends a device token to extend MFA authentication lifetime. This property is, by default, set to True. Set this to False only if you do not want MFA to be remembered.

Example connection string:

AuthScheme=Okta; AWSRegion=Ireland; User=user@cdata.com; Password=CH8WerW121235647iCa6; SSOLoginURL='https://cdata-us.okta.com/home/amazon_aws/0oa35m8arsAL5f5NrE6NdA356/272'; SSOProperties='ApiToken=01230GGG2ceAnm_tPAf4MhiMELXZ0L0N1pAYrO1VR-hGQSf;'; AWSRoleArn=arn:aws:iam::1234:role/Okta_SSO; AWSPrincipalARN=arn:aws:iam::1234:saml-provider/OktaProvider;

PingFederate

Set the AuthScheme to PingFederate. The following connection properties need to be set:

  • User: Set this to the PingFederate user.

  • Password: Set this to PingFederate password for the user.

  • SSOLoginURL: Set this to the login url used by the SSO provider.

  • SSOExchangeUrl: The Partner Service Identifier URI configured in your PingFederate server instance under: SP Connections > SP Connection > WS-Trust > Protocol Settings. This should uniquely identify a PingFederate SP Connection, so it is a good idea to set it to your AWS SSO ACS URL. You can find it under AWS SSO > Settings > View Details next to the Authentication field.

The following SSOProperties are needed to authenticate to PingFederate:

  • AuthScheme (optional): The authorization scheme to be used for the IdP endpoint. The allowed values for this IdP are None or Basic.

Additionally, you can use the following SSOProperties to configure mutual SSL authentication for SSOLoginURL, the WS-Trust STS endpoint:

  • SSLClientCert

  • SSLClientCertType

  • SSLClientCertSubject

  • SSLClientCertPassword

Below is an example connection string:

authScheme=pingfederate;SSOLoginURL=https://mycustomserver.com:9033/idp/sts.wst;SSOExchangeUrl=https://us-east-1.signin.aws.amazon.com/platform/saml/acs/764ef411-xxxxxx;user=admin;password=PassValue;AWSPrincipalARN=arn:aws:iam::215338515180:saml-provider/pingFederate;AWSRoleArn=arn:aws:iam::215338515180:role/SSOTest2;

MFA

For users and roles that require Multi-factor Authentication, specify the following to authenticate:

  • AuthScheme: Set this to AwsMFA.

  • CredentialsLocation: The location of the settings file where MFA credentials are saved. See the Credentials File Location page under Connection String Options for more information.

  • MFASerialNumber: The serial number of the MFA device if one is being used.

  • MFAToken: The temporary token available from your MFA device.

If you are connecting to AWS (instead of already being connected such as on an EC2 instance), you must additionally specify the following:

  • AWSAccessKey: The access key of the IAM user for whom MFA will be issued.

  • AWSSecretKey: The secret key of the IAM user whom MFA will be issued.

If you are also using an IAM role to authenticate, you must additionally specify the following:

  • AWSRoleARN: Specify the Role ARN for the role you'd like to authenticate with. This will cause the provider to attempt to retrieve credentials for the specified role using MFA.

  • AWSExternalId: Only if required when you assume a role in another account.

This causes the provider to submit the MFA credentials in a request to retrieve temporary authentication credentials.

Note that you can control the duration of the temporary credentials by setting the TemporaryTokenDuration property (default 3600 seconds).

Credentials Files

You can use a credentials file to authenticate. Any configurations related to AccessKey/SecretKey authentication, temporary credentials, role authentication, or MFA can be used. To do so, set the following properties to authenticate:

  • AuthScheme: Set this to AwsCredentialsFile.

  • AWSCredentialsFile: Set this to the location of your credentials file.

  • AWSCredentialsFileProfile (optional): Optionally set this to the name of the profile you would like to use from the specified credentials file. If not specified, the profile with the name default will be used.

AWS Cognito Credentials

If you want to use the provider with a user registered in a User Pool in AWS Cognito, set the following properties to authenticate:

  • AuthScheme: Set this to AwsCognitoSrp (recommended). You can also use AwsCognitoBasic.

  • AWSCognitoRegion: Set this to the region of the User Pool.

  • AWSUserPoolId: Set this to the User Pool Id.

  • AWSUserPoolClientAppId: Set this to the User Pool Client App Id.

  • AWSUserPoolClientAppSecret: Set this to the User Pool Client Secret.

  • AWSIdentityPoolId: Set this to the Identity Pool Id of the Identity Pool that is linked with the User Pool.

  • User: Set this to the username of the user registered in the User Pool.

  • Password: Set this to the password of the user registered in the User Pool.

PreviousAzure Table StorageNextIBM Cloud Object Storage

Last updated 2 years ago

AWSSessionToken: Your AWS session token. This will have been provided alongside your temporary credentials. See for more info.

See for more information.

AWS Identity and Access Management User Guide
AWS Command Line Interface User Guide