4.0.

...

10

2023-09-04

OECGI 4.0.2 Love

In older versions of the OECGI, if a developer wanted to capture the values of request headers that were not automatically included in the Request array, a comma delimited list of these headers needed to be entered into the Registry AdditionalValues string. As of OECGI 4.0.2, an asterisk character (*) can be entered instead. This puts all request headers into the Request array. All relevant services within the HTTP_Services module have been updated to be backwards and forward compatible.

Baking More Cookies

In v4.0.7 we provided a GetCookies and a GetCookie service in the HTTPClient_Services module. This was all well and good when handling API requests to other servers, but what about those cookies that were returned to your own APIs? Worry no more because we now have a GetCookies and GetCookie service in the HTTP_Services module to make it much easier to eat those tasty morsels.

What About Blob?

Sending a binary object to another server via a HTTP request doesn't always work. We have added a new argument, ResponseIsBinary, to the SendHTTPRequest service (a member of the HTTPClient_Services module) to force the a "blob" datatype. Note, this requires SRP Utilities 2.2.6 or higher.

4.0.9

2020-07-08

IP Banning

As a compliment to the existing Whitelisted IPs feature we now offer an option to Banned IPs:

Image Added

4.0.8

2020-03-04

OECGI4.php Love

Perhaps you don't know this, but Revelation Software provides two plug-ins for HTTP connectivity. The original and most likely used is OECGI4.exe (or OECGI3.exe for older systems). This is obviously designed for Windows web servers. The other version is OECGI4.php (and there is also an OECGI3.php). This version would be the plug-in of choice for non-Windows (usually Linux) based web servers. We installed the PHP version several years ago on a few sites and experienced no issues with it. However, a customer recently brought to our attention that their HTTP responses were coming back with the headers appearing with the body. Not good! Upon review and with some insight from the ever helpful Bryan Shumsky, we discovered that the PHP plug-in expects the response headers to be separated from the body with LF as the delimiter. The HTTP Framework has been using CR/LF as the delimiter all this time. There is a reason we use CR/LF delimiters...because this is what the HTTP specification requires. What we didn't know was that the PHP plug-in parses our response (expecting LF delimiters) and then builds a property response using PHP functions. Since we were using CR/LF delimiters, the parsing logic failed and kept the header and the body together. This isn't an issue for OECGI4.exe because it basically passes through the response "as is". Did you get all that? TL;DR: The HTTP Framework now checks to see if the EXE or PHP plug-in is being used and sends back the response using the correct delimiters.

4.0.7

2020-02-27

C is for Cookie, That's Good Enough For Me

Cookies have long been a valuable tool for maintaining state, implementing authentication, and tracking history with web applications. The SRP HTTP Framework has always supported cookies via the SetResponseHeaderField service (a member of the HTTP_Services module) and the GetResponseHeaderField service (a member of the HTTPClient_Services module). However, a recent project convinced us that Cookies needed a little first-class citizen love. Therefore we have added a few new services so baking and consuming cookies is much yummier:

• SetCookie (a member of the HTTP_Services module) - Adds a Set-Cookie header to the response using the indicated Name. The cookie's value and optional attributes will automatically be included as indicated by each argument.
• GetCookies (a member of the HTTPClient_Services module) - Returns all cookie strings from the response headers.
• GetCookie (a member of the HTTPClient_Services module) - Returns the value for the indicated cookie name.

4.0.6

2020-02-04

FrameWorks Friendlier

The SRP HTTP Framework has always been included with the full SRP FrameWorks package. However, the core forms (e.g., HTTP Framework Setup, Web Accounts, and HTTP Logs) were never designed with the intent of running in parallel with the main SRP FrameWorks MDI Frame. A client reported to us that after running our SRP HTTP Framework forms, their main MDI Frame stopped responding to clicks in the Ribbon control. We identified the problem being related to a memory cache conflict. SRP HTTP Framework now uses its own cache and so the two systems are playing nicely with each other again.

Managing Resources a Little Easier

The Resource Manager page of the HTTP Framework Setup form has a couple of new quality of life features:

• Resource Collapse. The hierarchical resource view can tend to get very busy when multiple resources are open. There are times when we just want to collapse everything to avoid excessive scrolling up and down. A new (fairly inconspicuous) button has been added to do just that:
  
  Image Added
  
  
• Resource Sorting. When new resources are added, they always land at the bottom. However, there are times when we simply want to view our resources in a different order than when their creation date. Now this can be accomplished with a simple drag-and-drop action (shame on us for not adding this sooner)!
  
  Image Added

Don't lose your HEAD

HEAD is one of the first methods introduced in the HTTP spec, but it is probably one of the lesser known members.  According to the specification, HEAD works exactly like GET but it does not return a body. So, why would anyone use it? First, it is a way to "test the waters" of a GET request without the overhead that this request might produce. Second, HEAD requests are supposed to return the same HTTP Response headers that a GET request, making it easier to anticipate how the body will be returned (e.g., the Content-Type header should still have a valid value, even if the body is empty). Finally, since the HEAD method requires authentication, it is a great way to "log into" an application without requiring the app to download and endpoint's resource right away. So it seemed time that the SRP HTTP Framework finally give some respect to this often overlooked method.

You may have noticed in the previous screen shots that the GET method displays the HEAD method next to it. This is to indicate that GET and HEAD methods will be enabled or disabled together. Furthermore, all API routines will automatically get a HEAD API signature added above the GET API signature:

////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////
// Endpoint Handlers
////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

API contacts.HEAD
API contacts.GET

    Method  = HTTP_Services('GetHTTPRequestMethod')
	// If the GET method then return a body. If the HEAD method then just return response headers.
    .
    .
    .
    .

end api

URL Templates and HAL

URI Templates (URL Templates going forward) were introduced in 2012 as a way of documenting the structure of a URL (including any query params). One primary purpose for URL Templates is for "the discovery of available services". This fits very well with the REST constraint of "resource identification". Simply put, a URL Template is an abstraction (or generic version) of a resource URL. A templated version of a URL contains arguments surrounded by curly braces like so:

https://www.examples.org/api/contacts/{contactID}

The HAL specification supports links to URL Templates (versus resource links). To help support URL Templates, the AddLinkRelation service (a member of HTTP_Resource_Services) has been updated with new arguments to make it easy to include a URL Template. Any valid resource URL within the system can be passed into the URL argument and if the IsTemplate argument is set to 1, then it will be converted to a URL Template. For example, the SRP HTTP Framework automatically ships with a Contacts resource which can be accessed via this URL:

https://www.examples.org/api/contacts

There are three query params supported by this resource: first_name, last_name, and state. If the above URL is passed into the AddLinkRelation service with the IsTemplate argument set to 1, then resulting URL will look like this:

https://www.examples.org/api/contacts{?first_name,last_name,state}

The act of converting a URL Template into a valid resource URL is called "expansion". This is something a consumer of your services might consider building. However, at a minimum, any human agent can review the URL Templates you provide and benefit from self-documenting APIs...which is a major objective of REST!

Embracing SRP Utilities 2.1

The most recent release of SRP Utilities includes many handy services. The SRP_DateTime function, in particular, has been incorporated into the SRP HTTP Framework. We are using it to replace all calls to Utility_DotNet("TIMEZONE") as a way of producing a UTC date/time stamp. We discovered that in some cases Utility_DotNet is unable to return the proper local time which results in a Get_Status() error...which then leads to 502 Bad Gateway errors. Ouch! Suffice to say that the SRP HTTP Framework 4.0.6 requires SRP Utilities 2.1.

4.0.5

More Logging TLC

The more logging gets used the more ways we see how this can be improved. Here is a summary of what we've added:

• To help speed up the population of the HTTP Logs utility, special log index files are automatically created. These files contain all of the data needed for the form and do not replace the individual log files. You can rebuild your log indexes at anytime.
  Image Added
• The HTTP Logs utility has a new column which displays the Remote Address (for requests) and the Execute Time (for responses).
  Image Added
• Searching logs no longer shrinks the list in the HTTP Logs utility but finds the next log matching the search criteria. Users can go forwards and backwards via newly added VCR style Find Next and Find Previous buttons.
  
  Image Added
• Logs before and for a specified date can now be archived via the Archive Logs menu. This feature zips up your logs and stores them in the archive sub-folder.
  
  Image Added

Image Automation

You might have already noticed that setting the Response body will often automatically set the Content-Type header field to the correct value (e.g., application/xml, application/json, text/html, etc.). This exists as a safety check for when developers forget to specify this. We've updated this feature to properly identify PNG, JPG, and GIF data formats.

Here's a Wild Idea...

• Non-Authenticated URLs now support wildcards:
  Image Added
  In the above example, an API developer can now support multiple OAuth redirect URLs (e.g., /oauth/google, /oauth/facebook, /oauth/microsoft, etc.) without having to list each one individually.
  
  
• Whitelisted IPs also support wildcards:
  Image Added
  In the above example, an API developer can now support a range of IPs rather than having to list each one individually.

It's a Bug Hunt

As always, we continue to squash bugs that come up in real world scenarios. Private William Hudson would have been proud.

4.0.4

Breaking the Log Jam

In our last release we added a feature so that individual endpoints can be excluded from logging. Still, the number of log files that get generated can still be daunting and difficult to sift through when specific issues need to be investigated. Never fear...the new HTTP Logs utility is here! Just launch the NDW_HTTP_LOGS form, confirm the log directory, and wonder why we never created this sooner!

Image Added

Paving the Road to Integration

Up until now, the Resource Manager feature of the HTTP Framework Setup form maintained exclusive control over resource endpoint definitions (read: tightly coupled code design). Recent projects necessitated a way for 3rd party utilities to tap into this same functionality. Rather than copy blocks of code, we decoupled the resource endpoint logic and relocated it into its own service module: HTTP_Resource_Manager_Services. This will be documented soon but, unless you have a need to manage endpoints outside of the HTTP Framework Setup form, you probably don't need to know about this.

4.0.3

Ping Me Up Sometime

A new Ping API has been added. Many systems need a way to monitor whether or not the API server is still running. The Ping API was created specifically for this purpose. It returns the current date and time in GMT format.

To Log, or Not to Log, that is the Question

Logging is great until you need to sort through thousands of logs that exist only to monitor the API server. Each resource endpoint now has the ability to be excluded from logging. The new Ping API is excluded by default.

Home URL on the Range

The HTTP Framework Setup provides a way to enter the expected Home URL for your APIs. This is used for logging and for producing hypermedia links in your API responses. However, this might not be useful for sites that support multiple URLs or IP addresses. Therefore, the Home URL will now be derived from the incoming request.

Time Off for Good Behavior

The new Web Accounts authentication system works really well at tracking the number of invalid password attempts. In fact, it works a little too well. It turns out that once an account has incurred an invalid password attempt, this incursion is remembered by the system even after the account successfully provides a valid password. This eventually causes an account to immediately become disabled after only one invalid password attempt. The system will now reset the number of invalid password attempts whenever a valid password is provided. The number of invalid passwords tracked by the overall system, however, is still accumulated until manually reset.

4.0.2

A Good Thing just gets Better

We hate to sound repetitive, but this is again a minor release that ties up more loose ends.

4.0.1

A Good Thing just gets Better

This release mostly ties up a few loose ends that were introduced in v4.0.0.

4.0.0

API Automation

Our next major release to the SRP HTTP Framework introduces some major improvements to the way resources and APIs are created. The HTTP Framework Setup form has been heavily extended to support new features. The most significant of these is the Resource Manager, a visual tool that allows developers to define endpoints and automatically generate API templates with the click of a few buttons!

Image Added

Authentication in a Box

While the SRP HTTP Framework has included support for HTTP Basic Authentication since its first release and it has made it easy to embrace other methods of authentication, it had never ship with an out-of-the box authentication management tool. We have rectified this by including a Web Accounts manager. Users can now be created with expiring passwords. The SRP HTTP Framework authentication service has been significantly updated to provide support for Web Accounts while making it even easier to support other methods as well. Also, unsuccessful authentication requests are now tracked and the system can be configured to automatically go into containment mode based on a predetermined number of invalid passwords.

Image Added

APIs are the new Services

As a way of embracing the awesomeness and convenience of our new enhanced BASIC+ syntax, we've added a new statement: GoToAPI. This effectively eliminates maintaining those strings of allowed methods and services in addition to the long and complicated Case statements to route each method and endpoint combination to an appropriate GoSub label. Seeing is believing...so check it out!

GoToAPI else
   // The specific resource endpoint doesn't have a API handler yet.
   HTTP_Services('SetResponseStatus', 204, 'This is a valid endpoint but a web API handler has not yet been created.')
end

Return Response OR ''

//////////////////////////

API Automation

Our next major release to the SRP HTTP Framework introduces some major improvements to the way resources and APIs are created. The HTTP Framework Setup form has been heavily extended to support new features. The most significant of these is the Resource Manager, a visual tool that allows developers to define endpoints and automatically generate API templates with the click of a few buttons!

Image Removed

Authentication in a Box

While the SRP HTTP Framework has included support for HTTP Basic Authentication since its first release and it has made it easy to embrace other methods of authentication, it had never ship with an out-of-the box authentication management tool. We have rectified this by including a Web Accounts module. Users can now be created with expiring passwords. The SRP HTTP Framework authentication service has been significantly updated to provide support for Web Accounts while making it even easier to support other methods as well. Also, unsuccessful authentication requests are now tracked and the system can be configured to automatically go into containment mode based on a predetermined number of invalid passwords.

Image Removed

APIs are the new Services

As a way of embracing the awesomeness and convenience of our new enhanced BASIC+ syntax, we've added a new statement: GoToAPI. This effectively eliminates maintaining strings of support services and the long and complicated Case statements that were required to route each HTTP method and endpoint combination to an appropriate GoSub label. Check it out!

GoToAPI else
   // The specific resource endpoint doesn't have a API handler yet.
   HTTP_Services('SetResponseStatus', 204, 'This is a valid endpoint but a web API handler has not yet been created.')
end

Return Response OR ''

//////////////////////////////////////////////////////////////////////////////////////////////////////////
// Endpoint Handlers
////////////
// Endpoint Handlers
/////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////////

API Contacts.POST

API contacts.POST
   HTTP_Resource_Services('PostDatabaseItem', 'CONTACTS', FullEndpointURL)
end api

API contacts.ID.PATCH
   KeyID   = EndpointSegment
   HTTP_Resource_Services('PatchDatabaseItem', 'CONTACTS', FullEndpointURL, KeyID)
end api

API contacts.ID.PUT
   KeyID   = EndpointSegment
   HTTP_Resource_Services('PostDatabaseItemPutDatabaseItem', 'CONTACTS', FullEndpointURL, KeyID)
end api

API contacts.ID.PATCHDELETE
   KeyID   =   = EndpointSegment
   HTTP_Resource_Services('PatchDatabaseItem', 'CONTACTS', FullEndpointURL, KeyID)
end api

API Contacts.ID.PUT
   KeyID   = EndpointSegment
   HTTP_Resource_Services('PutDatabaseItem', 'CONTACTS', FullEndpointURL, KeyID)
end api

API Contacts.ID.DELETE
   KeyID   = EndpointSegment
   HTTP_Resource_Services('DeleteDatabaseItem', 'CONTACTS', KeyID)
end apiEndpointSegment
   HTTP_Resource_Services('DeleteDatabaseItem', 'CONTACTS', KeyID)
end api

Due to the new syntax and the desire to maintain backwards compatibility with the legacy service code, the old format of HTTP_{Resource}_Services is now replaced with {Resource}_API.

Resource Building Your Way

The HTTP_Resource_Services routine has always been a convenient way to quickly convert OpenInsight data into rich JSON while following good RESTful practices such as including self-referencing and collection URIs. This is accomplished through a family of high-level services (e.g., GetDatabaseItem, PostDatabaseItem, DeleteDatabaseItem). However, there are times when a little more low-level effort is required. To help with this, several new services have been added to be a middle ground between these high-level services and direct calls to the SRP_JSON functionDue to the new syntax and the desire to maintain backwards compatibility with the legacy service code, the old format of HTTP_{Resource}_Services is now replaced with {Resource}_API.

3.0.4

Protecting Column Data

...