App Interface Concepts
Application Interface Concepts in Data Studio Connector - API, Authentication, Pagination and Rate Limiting
In Data Studio Connectors, application interface concepts are frequently used. These concepts can be listed as
And these are the most important terms in knowing functionality of Data Studio Connector. Knowing about all these concepts is important.
Lets understand each concept in brief.
What is API ?
APIs are the key component of our digital world and enables billions of experiences on every minute of every day.
API stands for Application Programming Interface. API is a software, that allows two applications to communicate each other. It means, API is the messenger that delivers you request to the provides that you are requesting it from and then delivers the response back to you.
APIs are unique in their functionalities and implementations. This allows implementations and definitions to vary without compromising each other.
How do APIs work?
APIs communicate through a set of rules that define how computers, machines and applications can talk to each other. These acts as a middle person between two applications that want to talk each other for a specific task.
A simple example to explain APIs working is, when you sign in to Facebook from your phone, you are telling the Facebook that, you want to access your account. So the mobile application makes call to API to get your Facebook account and credentials. Facebook then access this information from one of its servers and return the data to mobile application.
Companies use this technology to find more efficient ways to retrieve information faster to serve the customer.
All Data Studio Connectors internally make API call to retrieve the data into report.
Example of API call :
Lets see how the api call made from Data Studio Connector to get "Deals" object data from HubSpot CRM.
Once you connecting the Data Studio to HubSpot CRM through authorization, its internally considered as you giving permission to get data from your HubSpot account.
After Selecting object "Deals" from dropdown, and continuing with fields an api call is made to HubSpot CRM to get Data. The url internally constructed as in below image.
As a result of API call, it gets data from HubSpot CRM and provide it as response in report. The response will be as shown in following image.
What is Pagination?
Pagination is a process that is used to divide a large data into smaller discrete pages, and this process is also known as paging. Pagination is commonly used by web applications and can be seen on Google. When we search for something on Google, it shows the results on the separated page, this is the main idea of the pagination.
In Data Studio Connector application "Offset Pagination" is commonly used. Lets know about it.
Offset Pagination :
When offset pagination is used, then offset and limit will be the query parameters. These parameters used with API in pagination through data. Offset Pagination is used when items in data have fixed length.
In first call, to get data of first page, initially offset value will be zero. And the limit field is optional.
In fetching next page data, offset value will be sum of previous offset value and limit returned in the previous result. Its like previous offset + previous limit.
Offset and Limit :
All the Jivrus Data Studio connectors supports pagination and used the offset and limit values as per respective API documentation.
What is Authentication :
Authentication is the process of recognizing a user’s identity. It is the mechanism of associating an incoming request with a set of identifying credentials. The credentials provided are compared to those on a file in a database of the authorized user’s information on a local operating system or within an authentication server.
Why we need Authentication?
Authentication helps in providing security to the data source, also in securing networks. Authentication helps organization to keep their network secure, by allowing only authorized users to access to their resources. This may include websites, network based applications and databases.
Community connector authorization :
There are many community connectors and these are not developed by google. So if you want to use a connector, you need to authorize it by giving proper credentials. There are several methods which are used to authenticate. And those are as explained in the following.
How community connector authorization works?
When you authorize the community connector, which means you are providing permission to Data Studio to interact with connector on your behalf. Depending on the requirement of that connector, you are prompted to provide additional credentials to connector. So that it can access your data.
Authentication methods used by community connector are as follows.
OAuth 2.0 :
It is the standard protocol for authorization. Community connectors that use OAuth 2.0 present users with authorize button. Which helps in accessing connectors data through that protocol.
Sample Image of OAuth 2 authorization in Data Studio Connector is given as following.
Username and Password :
Connectors that require a username and password can request those through the community connector.
API Token :
Connectors may require you to enter an API token as an authentication method. This is a unique identifier supplied by the service to which you are connecting.
Rate Limiting :
A rate limiting is the number of API calls an app or user can make within a given time period. Rate Limiting is like both form of security and form of quality control. So rate limiting is integral part of APIs product growth and scalability.
Why rate limiting is needed?
Rate Limiting helps in various ways as listed below :
To run APIs efficiently as possible.
Its essential component of internet security, as it helps in avoiding unlimited API requests.
Helps to make API scalable.
How rate limit works?
Rate Limits acts as middleman in controlling outgoing and incoming traffic to or from network.
Lets consider an API with rate limit is 100 per second, if requests exceed this number, it will generate error message. In case of HTTP, API requests, this rate limit error give response code as 429.
Why you get rate limit error in Data Studio?
In most of the cases you may have faced the rate limit issue, in most of the connector. At that time you get the error message as "Sorry! we are using 'Urlfetch' service from Google, and looks like you have crossed the daily limit of service. For more details please visit Jivrus FAQ section: https://datastudio.jivrus.com/support/faqs/". Do you know why this error comes?
Google provides daily 20000 UrlFetch quota to each google account. This UrlFetch quota helps in user to make API call and getting data. So per day, each google account user get 20k UrlFetch quota. If your API calls exceeds this quota, you wont be able to make next API call for that day. So you need to wait for next day, till quota renewed. UrlFetch quota will be renewed daily based on your time zone.
How to handle Rate Limit error in Data Studio?
If you want to save quota, and dont want to get rate limit error, you need to avoid making unused API calls.
Since UrlFetch quota is related to your google account, all products, software , which use your google account will make use of daily quota. And it results in shortening API calls in Data Studio, and rate limit error occurs.
So make sure you are using only necessary products, which are useful in your daily work.