RESTful API Designing Guidelines

Your introduction for everything REST and how to understand it.

What is REST?

Representational state transfer (REST) is a software architectural style that defines a set of constraints to
be used for creating Web services. Web services that conform to the REST architectural style, called
RESTful Web services, provide interoperability between computer systems on the Internet.

Terminologies

The following are the most important terms related to REST APIs:

Resource
Resource is an object or representation of something, which has some associated data with it and
can be set methods to operate on it.
E.G Company

Collections
Collections are a set of resources.
E.G Companies is the collection of the Company Resource.

URL
URL (Uniform Resource Locator) is a path through which a resource can be located and actions can
be performed on it.

API Endpoint
In simple terms, an API endpoint is the point of entry in a communication channel where two systems are
interacting. It refers to a touch point of communication between an API and the server.
The location where the API sends a request and where the response emanates is what is known as an
endpoint.

API vs Endpoint
An API refers to a set of protocols and tools that allow interaction between two different applications. It is a technique that enables third-party vendors to write programs that can easily interface with each other. On the other hand, an endpoint is the place of interaction between applications. API refers to the whole set of protocols that allows communication between two systems while an endpoint is a URL that enables the API to gain access to resources on a server.

Web Api Endpoint Naming Convention

Methods Naming Convention
In OOP methods are named with verbs to make it easier to identify what operation that method will
perform, for example the method GetEmployees(int departmentID) will return all the employees that
belongs to a department.

Should we use the same naming convention when designing web Api’s endpoints?
The answer is NO.
Web Api endpoints must be names with NOUNS instead of verbs and it should contain the plural form of
the Resource the api will perform operations on.

Example: https://www.estradaci.com/apis/v1/Employees

If the URL can’t contain verbs, how do we know which action will be performed?
HTTP Verbs will be responsible for telling which action the WEB API should perform.
Let’s look at the most used HTTP Verbs used while creating WEB APIs.

HTTP VERBS.

GET
Use GET requests to retrieve resource representation/information only – and not to modify it in any
way. As GET requests do not change the state of the resource, these are said to be safe methods.
Additionally, GET APIs should be idempotent, which means that making multiple identical requests must
produce the same result every time until another API (POST or PUT) has changed the state of the
resource on the server.

Examples:
 GET https://www.estradaci.com/apis/v1/Employees
 GET https://www.estradaci.com/apis/v1/Employees/1
 GET https://www.estradaci.com/apis/v1/Employees?name=Jeff
All the endpoints above will fetch employees but using different inputs to query the employees.

HTTP POST
Use POST APIs to create new subordinate resources, e.g. a file is subordinate to a directory containing it
or a row is subordinate to a database table. Talking strictly in terms of REST, POST methods are used to
create a new resource into the collection of resources.
Examples:
 POST https://www.estradaci.com/apis/v1/Employees
 POST https://www.estradaci.com/apis/v1/Employees/Address

Both endpoints above will insert data in the database, the first one will create a new employee and the
second one will create an address for an employee.

HTTP PUT
Use PUT APIs primarily to update existing resource (if the resource does not exist then API may decide to
create a new resource or not). If a new resource has been created by the PUT API, the origin server
MUST inform the user agent via the HTTP response code 201 (Created) response and if an existing
resource is modified, either the 200 (OK) or 204 (No Content) response codes SHOULD be sent to
indicate successful completion of the request.
Examples:
 PUT https://www.estradaci.com/apis/v1/Employees
 PUT https://www.estradaci.com/apis/v1/Employees/Address

Both endpoints above will update data in the database, the first one will update an employee and the
second one will update the employee`s address.

HTTP DELETE
As the name applies, DELETE APIs are used to delete resources (identified by the Request-URI).
Examples:
 DELETE https://www.estradaci.com/apis/v1/Employees
 DELETE https://www.estradaci.com/apis/v1/Employees/Address

Both endpoints above will delete data from the database, the first one will delete an employee and the
second one will delete the employee`s address.
As we can see the same URL can perform different actions when requested with different HTTP Verbs.

For example the https://www.estradaci.com/apis/v1/Employees can Get, Create, Update and Delete
employees based on the HTTP Verb used.

HTTP response status codes

Sometimes when the client sends a request to the server it expects a response indicating the result of the
operation. it’s a good practice to return coherent status codes to make it easy to the client understand what
happened when the request is processed.
HTTP defines standard status codes that can be used to convey the results of a client’s request. The
status codes are divided into the five categories presented below.

CATEGORY DESCRIPTION

1xx: Informational Communicates transfer protocol-level information.

2xx: Success Indicates that the client’s request was accepted successfully.

3xx: Redirection Indicates that the client must take some additional action in order to complete their request.

4xx: Client Error This category of error status codes points the finger at clients.

5xx: Server Error The server takes responsibility for these error status codes.

Check below the description of the most used Status Codes used when creating a web api.

For a complete list of the HTTP Status Codes check https://docs.microsoft.com/en-
us/dotnet/api/system.net.httpstatuscode?view=netframework-4.8

200 (OK)
It indicates that the REST API successfully carried out whatever action the client requested, and that no
more specific code in the 2xx series is appropriate.

201 (Created)
A REST API responds with the 201 status code whenever a resource is created inside a collection. There
may also be times when a new resource is created as a result of some controller action, in which case
201 would also be an appropriate response.

204 (No Content)
The 204 status code is usually sent out in response to a PUT, POST, or DELETE request when the REST
API declines to send back any status message or representation in the response message’s body.
An API may also send 204 in conjunction with a GET request to indicate that the requested resource
exists, but has no state representation to include in the body.

304 (Not Modified)
This status code is similar to 204 (“No Content”) in that the response body must be empty. The key
distinction is that 204 is used when there is nothing to send in the body, whereas 304 is used when the
resource has not been modified since the version specified by the request headers If-Modified-Since or
If-None-Match.

400 (Bad Request)
400 is the generic client-side error status, used when no other 4xx error code is appropriate. Errors can
be like malformed request syntax, invalid request message parameters, or deceptive request routing
etc.
The client SHOULD NOT repeat the request without modifications.

401 (Unauthorized)
A 401 error response indicates that the client tried to operate on a protected resource without providing
the proper authorization. It may have provided the wrong credentials or none at all.

404 (Not Found)
The 404 error status code indicates that the REST API can’t map the client’s URI to a resource but may
be available in the future. Subsequent requests by the client are permissible.

500 (Internal Server Error)
500 is the generic REST API error response. Most web frameworks automatically respond with this
response status code whenever they execute some request handler code that raises an exception.

Content Negotiation

When sending a request to an API we need to tell the server what is the type of the data we are sending
and the server is responsible to tell the client the same.

At server side, an incoming request may have an entity attached to it. To determine it’s type, server
uses the HTTP request header Content-Type.

Some common examples of content types are :“text/plain”, “application/xml”, “text/html”, “application/json”, “image/gif”, and “image/jpeg”.

Content-Type: application/json
Similarly, to determine what type of representation is desired at client side, HTTP header ACCEPT is
used. It will have one of the values as mentioned for Content-Type above.
Accept: application/json

The most used content-type used by APIS to represent the object the is being sent to the server or
returned to the client is JSON, make sure to use the camelCase naming convention when using JSON.

API Versioning
One of the most important things in WEB API development is the versioning.
WEB APIs must be well versioned in order to prevent the clients that are consuming it to break.

When a Break Change is made to an existing WEB API, instead of modifying the existing one we must
create a new version of it.

For example:
Several changes are required to be made on the WEB API
https://www.estradaci.com/apis/v1/Employees and these changes may lead the consumers to break
their integrations.
Instead of simply applying the changes to this API we need to create a new version of the api, E.G
https://www.estradaci.com/apis/v2/Employees.
This will prevent the clients that are consuming the V1 to break and will give them the time and
flexibility to migrate their calls to the V2.

REST vs. gRPC – The Battle Over Service Orientation

Rest Vs. gRPC

Services are a common talk between developers and companies that are in the process of implementing software; they are an important piece in any project architecture as they separate layers of functionality and offer the flexibility of implementing different levels of security in between layers. Their implementation can vary from serving data from the database servers, sending messages to other servers, and much more.

In .NET technologies there used to exist an entire framework dedicated to services and Windows Communication Foundation, this collection of tools would help set up services in a variety of protocols. On the downside, they were closely coupled with .NET, making the configuration of services more complex than some of the most popular options of its time.

Then came REST, a new portable way of communicating clients and servers via TCP, making them accessible over the internet. Most of the big web sites implement REST APIs to provide information to the outside world (such as Google, Twitter, LinkedIn, etc.). This model is based on Request-Response model that will return a text-based response (JSON, XML, HTML), currently the standard is JSON (JavaScript Object Notation). This is a flexible way to work with objects without depending on the platform.

The Problem…

The problem with JSON comes when you have a high demand application (server or client) that will make a large amount of request/responses. If you have to serialize/deserialize the objects to your native types, it will add an extra conversion overhead. If you are not having a outword facing application, you mostly have the same technologies applied in both sides of the communication, meaning that the service will have to take the response object from C#, serialize it to JSON and send it back to the client. Then the client will take the response JSON and deserialize back to your C# object. This is by no means hard to achieve, but the processing time may be impacted by the workload of your application.

There is a solution!

To solve this, we have RPC (Remote Procedure Call). This communication is based on Protocol Buffers, a mechanism to serialize data (the data is serialized in raw format). It offers all the main functionalities that REST does, with the added option for bidirectional streams that allow the client and server to communicate asynchronously that result in faster times. The configuration steps to get a gRPC service may seem a little more complex and limited than REST (Protocol Buffers currently supports generated code in C++, Go, Java, Phyton, Ruby and C#), but the benefit of faster performance may compensate.

In conclusion.

Current implementation of services using gRPC came with the adoption of Cloud computing and microservices with high performance demand. So, if you are planning create or enhance a service take a look at the past first, then present technologies and the future of your implementation in order to make the correct call. In conclusion, REST is still the most simple and portable solution if you don’t have an issue with communication performance.

If you want to know more about gRPC go to https://grpc.io/docs/guides/

Happy 5 Year Anniversary Trijo!

November is a big month for ECI and it’s members;

We are happy to announce the 5 year anniversary of Trijo George at Estrada Consulting Inc.

Estrada Consulting Inc. is proud to have some of the greatest and most dedicated talent around as a part of its team. We know and appreciate the hard work and dedication that goes in to pulling off such a successful record of work across the industry.

We understand that you have a choice and we congratulate ourselves on having you a part of our team.

Thank you for bringing your best to help us achieve success across North America year after year.

From the ECI Family, we would like to congratulate you, and thank you on your 5 years anniversary of being a part of our story.

Congratulations on achieving this anniversary with us! We know you have worked hard for this accomplishment and we truly appreciate your dedication.

Project Approval Lifecycle Part 2

Miss part 1? Read it here ! 

This process requires that you take business requirements and translate them into detailed solution requirements.  In general, this process takes the business concepts and translates those into solution requirements that can be developed within the recommended alternative.   This process is intuitive for most project teams since it is typical to jump to a solution when a person or team is presented a problem.  However, just as the steps to document the business process was done iteratively to ensure all steps were included and provided a complete context to understand the problem, the process to progressively elaborate the solution requirements will take many iterations to completely define the technical requirements.  For example, if the mid-level business requirement is to send notifications when a triggering event occurs, this can translate into many detailed solution requirements to set up a service to send a notification and the rules that trigger the notification.    The Stage 3 document can take another 3 to 6 months to prepare for projects that are within the organization’s delegated purchasing authority and more time if the project requires a BCP.

Stage 3 Keys to Success

  • Spend adequate time, which can be anywhere from 3 to 12 months depending on the project scope, to define detailed solution requirements. The solution requirements will be the key input to develop the work breakdown structure and determine deliverable expectation documents for the team in charge of the project’s execution phase.
  • Develop the procurement vehicle based on CDT recommendations to ensure all aspects of the procurement have been considered and documented.

that are derived from the mid-level business requirements.  This process requires that you take business requirements and translate them into detailed solution requirements.  In general, this process takes the business concepts and translates those into technical requirements that can be developed in the recommended solution.   This process is intuitive for most project teams since it is typical to jump to a solution when a person or team is presented a problem.  However, just as the steps to document the business process was done iteratively to ensure all steps were included and provided a complete context to understand the problem, the process to progressively elaborate the technical requirements will require many iterations to completely define the technical requirements that convey what is needed to perform the business process step in the new solution.  For example if the mid-level business requirement is to send notifications when a triggering event occurs, this can translate into many detailed technical requirements to set up a service to send a notification with parameters to handle the many different situations when the notification service will be used.    The Stage 3 document can take another 3 to 6 months to prepare for projects that are within the organization’s delegated purchasing authority and more time if the budget requires a BCP and project oversight due to additional documentation requirements.

Stage 4 is the culmination of all the project planning and is the final check to ensure the business objectives have not changed since the Stage 1 analysis was completed, the mid-level requirements and finances gathered in Stage 2 remain consistent with organization’s needs and budget, and that the procurement methodologies and detailed requirements defined in Stage 3 have been prepared completely and are in alignment with prior analyses.  This final review is the last stage prior to issuing the procurement vehicle.  At this stage key action dates for the procurement are finalized, the planning phase is officially over, and the procurement begins.  The Stage 4 documentation consists of a final checklist to ensure the project team is ready to manage the tasks to address bidder questions, make any necessary changes as addenda to the procurement documents, evaluate the bidder responses, negotiate contract items, manage any protests and make the final decision to award.  Stage 4 documentation requires that each of these steps is documented and ensures the procurement was managed according to the plan.  All in all the completion of the stage gate PAL process can take anywhere from 9 months for mature organizations attempting to migrate a well-defined manual process to an electronic solution to 2 years for organizations which have little experience developing project plans or for large scale projects involving many business units across multiple systems.

If you have a project you want to explore more fully but need some help to get it off the ground, Estrada Consulting has excellent staff resources with considerable experience performing the work to complete the Project Approval Lifecycle process.  Our extensive experience in executing system development and implementation projects for government organizations will provide the insight to best prepare your organization to initiate and plan a successful project.

Project Approval Lifecycle Part 1

Project Approval Lifecycle 

Lessons from the California Department of Technology

Do you want to have better success implementing project deliverables?  Do you think you know how to solve a business problem but not sure how to initiate a project?

The CA Department of Technology (CDT) offers an excellent set of tools and techniques, that with practice, a dedicated project team and a deliberate effort will get you on your way to initiating and planning a successful project.

The tools I am referring to are the Project Approval Lifecycle (PAL) stage gate templates and instructions.  The techniques include focus groups and meetings with subject matter experts, key stakeholders and managers, depicting As-Is business processes using conceptual and process flow diagrams, and developing business narratives of the business problems and opportunities.

The PAL process defines the stages (stage gates), which if followed closely will help to move your project from just an idea at initiation to a formal plan for procurement, and the knowledge to support a successful project execution phase.  The goal for all projects is to have the foundation of the project so well defined, documented and understood that the execution phase will be in the best shape to handle technical complexities and unanticipated issues as they arise.

The PAL process employs four templates, which are named Stage 1 – Business Analysis, Stage 2 – Alternatives Analysis, Stage 3 – Solution Development and Stage 4 – Project Readiness and Approval.  The names themselves provide a high-level overview of the activities the project team will complete.  But don’t let the name fool you, there are many aspects that the project team must confront to produce quality analyses and acceptable documentation.

The first template,

Stage 1

– Business Analysis will have you focus on defining the project team, business background, stakeholders, business problems and/or opportunities and measurable objectives.  This seems like a simple list of items to tackle, but it’s not.  The first step in defining the project team is to identify staff who have the business knowledge and the willingness to take precious time out of their day to support a new effort which will not see benefits for some time.  These staff members must represent the entire process which will be affected by the eventual solution and be willing to work together to describe the current activities no matter how poorly each is executed.  The goal is to describe the current situation well enough that there is context for the team who will eventually help develop a solution.  If your audience does not understand the context no matter how bad the situation is, they will not be able to help solve the business problems and will not know how to help meet the business objectives.  This means that all staff who play a role in the business process will need to be represented to know you have captured the context fully.  When the project team does not know how to describe the context of the current situation the staff that is later employed to help solve the business problems will be in no place to help.  Estrada Consulting has been involved in many application development projects and can identify those organizations who have prepared themselves to tackle the difficult challenges of implementing a solution that hits the mark.  It takes the experience of senior business analysts to be able to coax the information from the project team and develop the project artifacts that go into the Stage 1 documentation.

For example, imagine you receive a business problem that states, “The business is not meeting its goal to complete customer requests within 4 business days”.  The corresponding objective may be: “Reduce the number of requests that are delayed due to missed communication by 20% within 6 months of solution implementation”.  This appears to be complete from a SMART (Specific, Measurable, Achievable, Realistic and Time Based) perspective, but there is not enough context for someone on the outside to understand how to help solve the business problem nor satisfy the objective.   The reader does not really understand why requests are being delayed.  The objective provides some insight that communications are being missed.  This seems simple to address by not missing any future communications.  We know that it is never this simple and if we had more context, we would know why these are missed and how to help.  If the business background description were complete we may find that the reason for missed communication is that the current process relies on email alone to communicate activities performed to complete a customer request.  Now the project team or an outside resource can begin to understand that the staff who manage customer requests are overwhelmed with email, which can be solved in a number of ways.  By knowing the context we can begin to determine if we have seen this problem before and what approaches were successful in similar situations in the past.  I may not have the right solution, but the community you rely on to provide solution alternatives will be able to provide a meaningful solution.

For mature organizations that have well-stated policies and procedures the answer to why the delay is occurring may be more easily determined by an outside person by performing a document review.  However, most organizations are facing new business problems and do not have these tools at their disposal.  As a departure from the CDT formula to define the As-IS Business process at the later Stage 2 – Alternatives Analysis, Estrada recommends for less mature organizations or those facing a new business problem, your organization analyze the business process fully before even documenting the business problems and objectives.  If the project team members can’t describe why they believe the problem exists and is not confident enough to put it in writing for the entire project team to see, we contend that the business problems are not completely known and the objectives when met will not have satisfied the needs of the organization.  It is easy to think you know why the business is encountering a problem, but without knowledge of all the actors and steps to complete the process, you will reliably miss important issues that will reduce the affects of the solution.

It is likely the project team will spend more time than expected to define the business background since it is extremely challenging to describe something that one does every day.  This is an iterative process to describe every step of the process and make sure the project team’s and stakeholder’s input is considered.  This is when an experienced business analyst can provide the most value to ensure that when conversations stall, the right probing questions are asked to keep the discussion moving along.  An additional benefit of doing the As-Is Business analysis at Stage 1 is that your team will learn the “hows and whys” each member does their work.  You will also find situations where you can make immediate process improvements that take little effort and will contribute to the success of the project.  This can create a synergy among the team where members will look to help each other and understand the trials of their co-workers.  The Stage 1 – Business Analysis along with the Stage 2 – Alternatives Analysis can often take the most time to complete within the PAL process, which it should.  Estrada Consulting recommends an organization spend 3 months at a minimum to complete the Stage 1 documentation.  For more mature organizations, that have documented the policies and procedures that describe the business background and context, this may only take weeks to complete.

Stage 2

focuses on the alternatives analysis and analyzing those against the current IT environment from the technical and financial perspectives.  Depending on the size and nature of the project and the number of stakeholders the S2AA can take as little as 3 to 6 months and as much as 6 to 12 months to complete.

The S2AA begins with defining the mid-level business requirements that can be derived from analyzing the AS-Is business process flows and narratives, and drive the market research.  Having a clear understanding of the business background, problems and mid-level business requirements is paramount to conducting quality market research.  This knowledge will allow the organization to clearly describe to your audience what the business needs are and how you plan to measure success through SMART objectives.  No matter the type of research the organization plans to conduct from research on the Internet to a formal Request for Information (RFI), the research analysis and assessment team will have the criteria to evaluate proposed alternative solutions.  One of the biggest returns on the market research investment will be quality analysis of the advantages/disadvantages of each alternative, which will be based on well-defined evaluation criteria.  The market research will also help the project team understand what is available out in the market and the cost of each alternative.  This knowledge can often translate into more meaningful requirements that can be prioritized.  The requirements become more meaningful since they can be evaluated and categorized more easily as mandatory versus optional and based on the market alternatives, the most important requirements begin to surface among the team as it argues for the features in each solution.

Another important aspect of the S2AA is to document the financial impact of the current environment and associate a value to the products and services provided by the organization.  The financial analysis will allow the project team to document the added costs (if applicable) to determine if the benefits in implementing the solution can be justified in efficiencies gained or when business growth can be better managed.  For government projects the goal isn’t typically to reduce costs associated with staffing levels, rather to improve efficiencies to address growing demand.  Therefore, the Financial Analysis should not focus for government organizations on reducing costs as the primary focus, rather to better understand what the new normal will cost against current expenditures to provide the costs that are associated with growth in the business when constituents ask how money is being spent and why.  Information technology projects can be expensive and typically cost more than what has been planned when the work to complete these two stages is not done properly.  Most projects that look to implement an electronic solution for a paper-based, manual process that can no longer sustain the business needs will likely increase operational costs and total cost of ownership but will ensure the likelihood of efficiencies as the business continues to grow.  When the organization can rely on the cost estimates at this stage, it will be better prepared to adjust to the new costs and will not be surprised with an out of control budget as the project progresses through the execution phase.

Once the organization has completed the S2AA and has determined final requirements, calculated a budget and identified a recommended alternative solution, the difficult work to define detailed solution requirements comes into play.  This is the Stage 3 – Solution Development work, in which the detailed requirements are derived from the mid-level business requirements and the project team decides on the approach and methodologies to procure goods and services.  Depending on the estimated costs for the project the organization may need to also prepare a Budget Change Proposal to acquire additional funding.  In any project that is estimated to surpass the organizations delegated purchasing authority and/or requires a BCP, there are additional forms and oversight that CDT will require while completing the Stage 3 and 4 documents and on into the execution phase of the project.  CDT offers an extensive resource to draft the procurement vehicle which describes each section that should be considered based on experience from previous implementation work, and an explanation that describes the content that should be included.  During Stage 3 most time will be spent defining detailed solution requirements

Continue Reading Part 2 

1 2 3 5