We've recently moved hosts! Please report any weirdness with the wiki (or spam) on Utopia.

NuDOC Protocol RFC

From BBSWiki

NOTE: This draft is now effectively obsolete, and will likely be replaced with a much simpler and less reinvented-wheel protocol based on XML-RPC, JSON, or similar.

Request for Comments
Category: Informational

February 2007

An XML-Based Protocol for Next-Generation DOC (nuDOC)

This document presents a draft standard for the protocol used between bulletin board system (BBS) clients and servers from the Next-Generation DOC (nuDOC) family. It includes XML Schema definitions and a description of how they should be used.

1 Protocol
2 Security Considerations

Protocol

Connections may be either stateful (a constantly maintained TCP connection from beginning to end of the user's session) or stateless (sequential TCP requests, one per connection). Clients using stateless connections must poll the server for broadcast type events if such clients choose to support them.

"Stateless" connections do maintain state on either end. In stateless connections, the server offers the client a token during session setup which the client then presents with each request. This functionality is currently unimplemented and is not supported in the current XML Schema.

Each message passed over the wire is a complete XML document conforming to a specific XML Schema Definition. The XML documents are encoded and sent using Netstrings. Netstrings are a very simple, 8-bit clean method of transferring data specified by D. J. Bernstein in 1997. The payload string is prepended with its length in bytes specified as a decimal number (represented in ASCII) plus a colon, and has a comma tacked on the end. A complete writeup is available at http://cr.yp.to/proto/netstrings.txt -- a simple example would be the string "hello world!" which is encoded as "12:hello world!," as a Netstring.

Upon connecting to the server, the client sends a handshake message listing the protocol XML Schemas and connection options (such as gzip) that it supports. The server then responds with a message either acknowledging some combination of protocol XML Schema and connection options or refusing the client's connection. This handshaking process is governed by an XML Schema contained in this document which must be fully supported by every client and server.

After client and server have agreed upon an XML Schema and a set of connection options, they transition to using that XML Schema and begin exchanging data.

Handshaking

Upon connection, client and server handshake with a series of XML documents conforming to a basic XML Schema definition which both are guaranteed to support. This basic XML schema describes how to negotiate various connection options such as which protocol XML Schema definition to use, whether to use gzip compression, and so forth. All servers and clients must support this basic XSD in addition to whatever versions of the protocol XSDs they support for actual exchange of data.

The client sends a handshake message upon its initial connection. This message contains information about the connection type (stateful or stateless), the protocol definitions supported, and the client's preferences for various connection options. The protocol definitions supported are listed in order of preference, with the first one being the most preferred and the last one being the least preferred.

The server then responds with a message containing either a NAK or, if negotiation is successful (the server and client have at least 1 protocol definition in common), a message with the best available protocol definition among the client's choices, and the values the server agrees to for connection options. The server's word is final. After the response, the client must either acknowledge, at which point both server and client transition to using the agreed-upon protocol definition, or the client must NAK and disconnect.

With a positive response, the server also includes a nonce, or random integer, which the client must present an incremented version of with each subsequent request. This sets the base value of the nonce that will be used in all non-broadcast traffic between client and server; see below for further details.

Normal Operation

After transitioning from the handshake phase to an agreed-upon protocol XSD, the server sends an initial "response" which serves as a login prompt. After this, the client generally makes requests which are followed by server responses. All requests and responses are complete XML documents which must validate with the agreed-upon protocol Schema.

Occasionally, such as with the receipt of an instant message, the server will push, or broadcast, an XML document without any request by the (stateful) client. The client must acknowledge receipt of these broadcasts.

With each response, the server sends a nonce, and with each subsequent request it will expect the nonce to be returned, incremented, by the client.

XML Schema Definitions

Handshaking

Below is a human-readable diagram of the handshake XSD along with the complete XML Schema.

And here is a link to the schema: conspire-handshake
And in raw form: http://bbswiki.evernex.com/wiki/Conspire-handshake?action=raw

<?xml version="1.0"?>
<xs:schema xmlns:xs="http://www.w3.org/2001/XMLSchema" xmlns="http://bbswiki.evernex.com/wiki/NuDOC" targetNamespace="http://bbswiki.evernex.com/wiki/NuDOC" elementFormDefault="qualified">
 <xs:element name="Handshake">
   <xs:complexType>
     <xs:sequence>
       <xs:element name="ConnectionNonce" type="xs:int">
         <xs:annotation>
           <xs:documentation>Upon server initial ACK, a ConnectionNonce is given which must then be incremented and presented by the client in every subsequent request. The nonce must be present but its value is ignored in the client's first message to the server.</xs:documentation>
         </xs:annotation>
       </xs:element>
       <xs:element name="ControlMessage">
         <xs:simpleType>
           <xs:restriction base="xs:string">
             <xs:enumeration value="REQ"/>
             <xs:enumeration value="ACK"/>
             <xs:enumeration value="NAK"/>
           </xs:restriction>
         </xs:simpleType>
       </xs:element>
       <xs:element name="ConnectionType" minOccurs="0">
         <xs:simpleType>
           <xs:restriction base="xs:string">
             <xs:enumeration value="stateful"/>
             <xs:enumeration value="stateless"/>
           </xs:restriction>
         </xs:simpleType>
       </xs:element>
       <xs:element name="ProtocolSupport" minOccurs="0">
         <xs:annotation>
           <xs:documentation>Minimum of 1, maximum of 255 ProtocolDefs as children of a ProtocolSupport element.  Server will only ever send one, the one it chooses; client may send up to 255 for the server to choose from.</xs:documentation>
         </xs:annotation>
         <xs:complexType>
           <xs:sequence>
             <xs:element name="ProtocolDef" type="xs:string" maxOccurs="255"/>
           </xs:sequence>
         </xs:complexType>
       </xs:element>
       <xs:element name="ConnectionOptions" minOccurs="0">
         <xs:annotation>
           <xs:documentation> Min of 0, max of 255 connection options, name attribute required </xs:documentation>
         </xs:annotation>
         <xs:complexType>
           <xs:sequence>
             <xs:element name="Option" minOccurs="0" maxOccurs="255">
               <xs:complexType>
                 <xs:simpleContent>
                   <xs:extension base="xs:string">
                     <xs:attribute name="name" type="xs:string" use="required"/>
                   </xs:extension>
                 </xs:simpleContent>
               </xs:complexType>
             </xs:element>
           </xs:sequence>
         </xs:complexType>
       </xs:element>
     </xs:sequence>
   </xs:complexType>
 </xs:element>
</xs:schema>

Example

Upon client connection, the client may send something like the following. Note the ConnectionNonce of 0. This is a placeholder; the server sets the nonce upon its first reply.

<?xml version="1.0" encoding="UTF-8"?>
<Handshake
 xmlns="http://www.smartcatmedia.com/nudoc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.smartcatmedia.com/nudoc handshake.xsd">
  <ConnectionNonce>0</ConnectionNonce>
  <ControlMessage>REQ</ControlMessage>
  <ConnectionType>stateful</ConnectionType>
  <ProtocolSupport>
    <ProtocolDef>http://www.smartcatmedia.com/nudoc nudoc-0.8.xsd</ProtocolDef>
    <ProtocolDef>http://www.smartcatmedia.com/nudoc nudoc-1.0.xsd</ProtocolDef>
    <ProtocolDef>http://www.alternatecoder.com/ alt-nudoc-02.xsd</ProtocolDef>
  </ProtocolSupport>
  <ConnectionOptions>
    <Option name="gzip">true</Option>
    <Option name="deflate">true</Option>
    <Option name="server-keepalive">300</Option>
  </ConnectionOptions>
</Handshake>

If the server does not support any of the protocolDefs requested by the client, the server would send something like this. Note the pointless but included ConnectionNonce.

<?xml version="1.0" encoding="UTF-8"?>
<Handshake
 xmlns="http://www.smartcatmedia.com/nudoc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.smartcatmedia.com/nudoc handshake.xsd">
  <ConnectionNonce>398283</ConnectionNonce>
  <ControlMessage>NAK</ControlMessage>
</Handshake>

If the server does support at least one of the requested ProtocolDefs, the response might look like this with a particular ProtocolDef and set of ConnectionOptions chosen. Note the changed value for deflate in the connectionOptions -- the server is refusing the deflate option and its word is final.

<?xml version="1.0" encoding="UTF-8"?>
<Handshake
 xmlns="http://www.smartcatmedia.com/nudoc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.smartcatmedia.com/nudoc handshake.xsd">
  <ConnectionNonce>569043</ConnectionNonce>
  <ControlMessage>ACK</ControlMessage>
  <ConnectionType>stateful</ConnectionType>
  <ProtocolSupport>
    <ProtocolDef>http://www.smartcatmedia.com/nudoc nudoc-1.0.xsd</ProtocolDef>
  </ProtocolSupport>
  <ConnectionOptions>
    <Option name="gzip">true</Option>
    <Option name="deflate">false</Option>
    <Option name="server-keepalive">300</Option>
  </ConnectionOptions>
</Handshake>

If the client accepts -- note the incremented nonce:

<?xml version="1.0" encoding="UTF-8"?>
<Handshake
 xmlns="http://www.smartcatmedia.com/nudoc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.smartcatmedia.com/nudoc handshake.xsd">
  <ConnectionNonce>569044</ConnectionNonce>
  <ControlMessage>ACK</ControlMessage>
</Handshake>

If not, the client sends a NAK and disconnects:

<?xml version="1.0" encoding="UTF-8"?>
<Handshake
 xmlns="http://www.smartcatmedia.com/nudoc"
 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
 xsi:schemaLocation="http://www.smartcatmedia.com/nudoc handshake.xsd">
  <ConnectionNonce>569044</ConnectionNonce>
  <ControlMessage>NAK</ControlMessage>
</Handshake>

At this point, if the client accepted, the server and client would transition to the agreed-upon XSD and begin transferring data with it.

Normal Operation

During post-handshake operation, the communication model is one of client request followed by server response, interspersed with occasional server broadcast (push) messages which the client must acknowledge receiving. Well-formed messages must validate against the protocol XML Schema, which is not included in this document for brevity; instead, human-readable diagrams of the major modes of communication are included.

To view the entire XML Schema, please visit:

A human-readable diagram of the extremely high-level architecture of the messages, or XML documents, passed between client and server follows: