1.2.1.12 HTTP Module

The HTTP Web Server module allows a development board to act as a Web server. This facilitates an easy method to view status information and control applications using any standard Web browser.

Note: The HTTP module is deprecated and it is maintained for backward compatibility only. It will be eventually removed from the TCP/IP Stack. New designs should use the HTTP_NET module.

Abstraction Model

Three main components are necessary to understand how the HTTP web server works: the web pages, the MPFS2 Utility, and the source files custom_http_app.c and http_print.h. An overview of the entire process is shown in the following diagram.

• Web Pages

  This includes all the HTML and associated images, CSS style sheets, and JavaScript files necessary to display the website. A sample application including all these components is located in the web_pages folder.

• MPFS2 Utility

  This program, supplied by Microchip, packages the web pages into a format that can be efficiently stored in either external non-volatile storage, or internal Flash program memory. This program also indexes dynamic variables found in the web pages and updates http_print.h with these indices. The MPFS2 Utility generates a SYS_FS compatible file image that can be directly uploaded on the development board or stored in the Flash program memory as a source file image to be included in the project.

  When dynamic variables are added or removed from your application, the MPFS2 Utility will update http_print.h. When this happens, the project must be recompiled in the MPLAB X IDE to ensure that all the new variable indices get added into the application.

• custom_http_app.c

  This file implements the Web application. It describes the output for dynamic variables (via TCPIP_HTTP_Print_varname callbacks), parses data submitted through forms (in TCPIP_HTTP_GetExecute and TCPIP_HTTP_PostExecutePost) and validates authorization credentials (in TCPIP_HTTP_FileAuthenticate and TCPIP_HTTP_UserAuthenticate).

  The functionality of these callback functions is described within the demonstration application's web pages, and is also documented within the custom_http_app.c example that is distributed with the stack.

• http_print.h

  This file is generated automatically by the MPFS2 Utility. It indexes all of the dynamic variables and provides the "glue" between the variables located in the Web pages and their associated TCPIP_HTTP_Print_varname callback functions defined in custom_http_app.c. This file does not require modification by the programmer.

HTTP Features

Dynamic Variables

Basic Use

To create a dynamic variable, simply enclose the name of the variable inside a pair of tilde (~) characters within the web pages' HTML source code. (ex: _myVariable) When you run the MPFS2 Utility to generate the web pages, it will automatically index these variables in http_print.h. This index will instruct your application to invoke the function TCPIP_HTTP_Print_myVariable when this string is encountered.

Here is an example of using a dynamic variable to insert the build date of your application into the web pages:

<div class="examplebox code">~builddate~</div>
The associated callback will print the value into the web page:
void TCPIP_HTTP_Print_builddate(HTTP_CONN_HANDLE connHandle)
{
    TCP_SOCKET sktHTTP = TCPIP_HTTP_CurrentConnectionSocketGet(connHandle);
    TCPIP_TCP_StringPut(sktHTTP, (const void*)__DATE__" "__TIME__);
}

Passing Parameters

You can also pass parameters to dynamic variables by placing numeric values inside of parenthesis after the variable name.

For example, _led(2) will print the value of the second LED. The numeric values are passed as 16 bit values to your callback function. You can pass as many parameters as you wish to these functions, and if your C code has constants defined, those will be parsed as well. (ex: _pair(3,TRUE)).

The following code inserts the value of the push buttons into the web page, all using the same callback function:

<div class="examplebox code">btn(3)~ btn(2)~ btn(1)~ btn(0)~</div>
This associated callback will print the value of the requested button to the web page:
void TCPIP_HTTP_Print_btn(HTTP_CONN_HANDLE connHandle, uint16_t num)
{
    // Determine which button
    switch(num)
    {
        case 0:
            num = BUTTON0_IO;
            break;
        case 1:
            num = BUTTON1_IO;
            break;
        case 2:
            num = BUTTON2_IO;
            break;
        default:
            num = 0;
    }

    // Print the output
    TCPIP_TCP_StringPut(TCPIP_HTTP_CurrentConnectionSocketGet(connHandle), (num ? "up" : "down"));
}

Longer Outputs

The HTTP protocol operates in a fixed memory buffer for transmission, so not all data can be sent at once. Care must be taken inside of your callback function to avoid overrunning this buffer.

The callback functions must check how much space is available, write up to that many bytes, and then return. The callback will be invoked again when more space is free.

Note: For increased throughput performance, it is recommended that the callback should always try to write as much data as possible in each call.

To manage the output state, callbacks should make use of the callbackPos variable that is maintained by the Web server for each individual connection. This 32-bit value can be manipulated with the TCPIP_HTTP_CurrentConnectionCallbackPosGet and TCPIP_HTTP_CurrentConnectionCallbackPosSet functions.

The callbackPos is set to zero when a callback is first invoked. If a callback is only writing part of its output, it should set this field to a non-zero value to indicate that it should be called again when more space is available. This value will be available to the callback during the next call, which allows the function to resume output where it left off. A common use is to store the number of bytes written, or remaining to be written, in this field. Once the callback is finished writing its output, it must set callbackPos back to zero to indicate completion.

The following code outputs the value of the build date of the application into the web pages:

<div class="examplebox code">~builddate~</div>

void TCPIP_HTTP_Print_builddate(HTTP_CONN_HANDLE connHandle)
{
    TCP_SOCKET sktHTTP;
    sktHTTP = TCPIP_HTTP_CurrentConnectionSocketGet(connHandle);

    TCPIP_HTTP_CurrentConnectionCallbackPosSet(connHandle, 0x01);
    if(TCPIP_TCP_PutIsReady(sktHTTP) < strlen((const char*)__DATE__" "__TIME__))
        return;

    TCPIP_HTTP_CurrentConnectionCallbackPosSet(connHandle, 0x00);
    TCPIP_TCP_StringPut(sktHTTP, (const void*)__DATE__" "__TIME__);
}

The initial call to TCPIP_TCP_PutIsReady determines how many bytes can be written to the buffer right now. When there is not enough buffer space for performing the output operation the function sets the current value of the callbackPos and returns. Once the output operation is performed the callbackPos is cleared to '0'.

Including Files

Often it is useful to include the entire contents of another file in your output. Most web pages have at least some portion that does not change, such as the header, menu of links, and footer. These sections can be abstracted out into separate files which makes them easier to manage and conserves storage space.

To include the entire contents of another file, use a dynamic variable that starts with "inc:", such as _{inc:header.inc}.

This sequence will cause the file header.inc to be read from the file system and inserted at this location.

The following example indicates how to include a standard menu bar section into every page:

<div id="menu">~inc:menu.inc~</div>

At this time, dynamic variables are not recursive, so any variables located inside files included in this manner are not parsed.

Form Processing

The GET Method

The GET method appends the data to the end of the URI. This data follows the question mark (?) in the browser's address bar. (ex: http://mchpboard/form.htm?led1=0&led2=1&led3=0) Data sent via GET is automatically decoded and stored in the current connection data buffer. Since it is to be stored in memory, this data is limited to the size of the connection data buffer which by default is 100 bytes (configurable by HTTP_MAX_DATA_LEN build time symbol or by the run time data passed at the HTTP module initialization).

It is generally easier to process data received in this manner.

The callback function TCPIP_HTTP_GetExecute is implemented by the application developer to process this data and perform any necessary actions. The function TCPIP_HTTP_ArgGet provides an easy method to retrieve submitted values for processing.

See the custom _http_app.c for an example of TCPIP_HTTP_GetExecute implementation.

The POST Method

The POST method transmits data after all the request headers have been sent. This data is not visible in the browser's address bar, and can only be seen with a packet capture tool. It does however use the same URL encoding method.

The HTTP server does not perform any preparsing of this data. All POST data is left in the TCP buffer, so the custom application will need to access the TCP buffer directly to retrieve and decode it. The functions TCPIP_HTTP_PostNameRead and TCPIP_HTTP_PostValueRead have been provided to assist with these requirements. However, these functions can only be used when at least entire variables are expected to fit in the TCP buffer at once. Most POST processing functions will be implemented as state machines to use these functions. There is a status variable per each connection that stores the current state accessible with TCPIP_HTTP_CurrentConnectionPostSmGet and TCPIP_HTTP_CurrentConnectionPostSmSet functions. This state machine variable is reset to zero with each new request. Functions should generally implement a state to read a variable name, and another to read an expected value. Additional states may be helpful depending on the application.

The following example form accepts an e-mail address, a subject, and a message body. Since this data will likely total over the size of the internal connection data buffer, it should be submitted via POST.

<form method="post" action="/email.htm">
To: <input type="text" name="to" maxlength="50" /><br />
Subject: <input type="text" name="subject" maxlength="50" /><br />
Message:<br />
<textarea name="msg" rows="6"></textarea><br />
<input type="submit" value="Send Message" /></div>
</form>

Suppose a user enters the following data into this form:

To: joe@picsaregood.com 
Subject: Sent by a PIC 
Message: I sent this message using my development board! 

The TCPIP_HTTP_PostExecute function will be called with the following data still in the TCP buffer:

to=joe%40picsaregood.com&subject=Sent+by+a+PIC&msg=I+sent+this+message+using+my+development+board%21 

To use the e-mail module, the application needs to read in the address and the subject, store those in RAM, and then send the message. However, since the message is not guaranteed to fit in RAM all at once, it must be read as space is available and passed to the e-mail module. A state machine, coupled with the TCPIP_HTTP_PostNameRead and

TCPIP_HTTP_PostValueRead functions can simplify this greatly.

See the TCPIP_HTTP_PostExecute and HTTPPostEmail functions in the supplied custom_http_app.c for a complete implementation of this example.

Authentication

Authentication functionality is supported by two user-provided callback functions. The first, TCPIP_HTTP_FileAuthenticate, determines if the requested page requires valid credentials to proceed. The second, TCPIP_HTTP_UserAuthenticate, checks the user name and password against an accepted list and determines whether to grant or deny access. The connection stores the current authorization setting which can be manipulated by using the TCPIP_HTTP_CurrentConnectionIsAuthorizedGet and TCPIP_HTTP_CurrentConnectionIsAuthorizedSet functions.

Note: The HTTP authentication is NOT secure! The passwords are sent in clear over the network. For a secure approach, the HTTP_NET module with TLS enabled should be used.

Requiring Authentication

When a request is first made, the function TCPIP_HTTP_FileAuthenticate is called to determine if that page needs password protection. This function returns a value to instruct the HTTP server how to proceed. The most significant bit indicates whether or not access is granted. That is, values 0x80 and higher allow access unconditionally, while values 0x79 and lower will require a user name and password at a later point. The value returned is stored in the connection data so that it can be accessed by future callback functions.

The following example is the simplest case, in which all files require a password for access:

uint8_t TCPIP_HTTP_FileAuthenticate(HTTP_CONN_HANDLE connHandle, uint8_t* cFile)
{
return 0;
}

In some cases, only certain files will need to be protected. The second example requires a password for any file located in the /treasure folder:

uint8_t TCPIP_HTTP_FileAuthenticate(HTTP_CONN_HANDLE connHandle, uint8_t* cFile)
{
    // Compare to "/treasure" folder. Don't use strcmp here, because
    // cFile has additional path info such as "/treasure/gold.htm"
    if(memcmp(cFile, (const void*)"treasure", 8) == 0)
    {   // Authentication will be needed later
        return 0;
    }

    // No authentication required
    return 0x80;
}

More complex uses could require an administrative user to access the /admin folder, while any authenticated user can access the rest of the site. This requires the TCPIP_HTTP_FileAuthenticate to return different authentication values for a different file.

Validating Credentials

The TCPIP_HTTP_UserAuthenticate function determines if the supplied user name and password are valid to access the specific resource. Again, the most significant bit indicates whether or not access is granted. The value returned is also stored in the connection internal data and it can be accessed by future callback functions.

The following example is the simplest case, in which one user/password pair is accepted for all pages:

uint8_t TCPIP_HTTP_UserAuthenticate(HTTP_CONN_HANDLE connHandle, uint8_t* cUser, uint8_t* cPass)
{
    if(!strcmp((char*)cUser, (const char*)"AliBaba") &&
    !strcmp((char*)cPass, (const char*)"Open Sesame!") )
    {
        return 0x80;
    }

    return 0;
}

More complex uses are certainly feasible. Many applications may choose to store the user names and passwords in protected files so that they may be updated by a privileged user. In some cases, you may have multiple users with various levels of access. The application may wish to return various values above 0x80 in TCPIP_HTTP_UserAuthenticate so that later callback functions can determine which user logged in.

Cookies

Cookies were added to the protocol description to solve this problem. This feature allows a web server to store small bits of text in a user's browser. These values will be returned to the server with every request, allowing the server to associate session variables with a request. Cookies are typically used for more advanced authentication systems.

Best practice is generally to store the bulk of the data on the server, and store only a unique identifier with the browser. This cuts down on data overhead and ensures that the user cannot modify the values stored with the session. However, logic must be implemented in the server to expire old sessions and allocate memory for new ones. If sensitive data is being stored, it is also important that the identifier be random enough so as to prevent stealing or spoofing another user's cookies.

Retrieving Cookies

In the HTTP server, cookies are retrieved automatically. They are stored in the connection internal data buffer just as any other GET form argument or URL parameter would be. The proper place to parse these values is therefore in the TCPIP_HTTP_GetExecute callback using the TCPIP_HTTP_ArgGet.

This model consumes some of the limited space available for URL parameters. Ensure that cookies do not consume more space than is available (as defined by HTTP_MAX_DATA_LEN) and that they will fit after any data that may be submitted via a GET form. If enough space is not available, the cookies will be truncated.

Setting Cookies

Cookies can be set in TCPIP_HTTP_GetExecute or TCPIP_HTTP_PostExecute. To set a cookie, store the name/value pairs in the connection internal buffer data as a series of null-terminated strings. Then, call TCPIP_HTTP_CurrentConnectionHasArgsSet with a parameter equal to the number of name/value pairs to be set. For example, the following code sets a cookie indicating a user's preference for a type of cookie:

void TCPIP_HTTP_GetExecute(void)
{
...
// Set a cookie
uint8_t* connData = TCPIP_HTTP_CurrentConnectionDataBufferGet(connHandle);
uint16_t httpBuffSize  = TCPIP_HTTP_CurrentConnectionDataBufferSizeGet(connHandle);
if(strlen(“flavor”) + strlen(“oatmeal raisin) + 2 <= httpBuffSize)
    {
    strcpy((char*)connData, "flavor");
    strcpy((char*)connData + strlen("flavor") + 1, "oatmeal raisin");

    TCPIP_HTTP_CurrentConnectionHasArgsSet(connHandle, 1);
    }

...
}

After this, all future requests from this browser will include the parameter "flavor" in the connection data along with the associated value of "oatmeal raisin".

Compression

The MPFS2 Utility will automatically determine which files can benefit from GZIP compression, and will store the compressed file in the MPFS2 image when possible. This generally includes all JavaScript and CSS files. (Images are typically already compressed, so the MPFS2 Utility will generally decide it is better to store them uncompressed.) This HTTP server will then seamlessly return this compressed file to the browser. Less non-volatile storage space will be required for storing the pages image, and faster transfers back to the client will result. No special configuration is required for this feature.

To prevent certain extensions from being compressed, use the Advanced Settings dialog in the MPFS2 Utility.

Library Interface