Defining Generalized Distributed Computing

There’s probably a lot of confusion about what I believe I’m accomplishing - what the goal of MAD is. This post is going to be somewhat more formal (in the sense of logic), if probably not up to academic or professional standards.

Let us define the problem of “Generalized Distributed Computing”. The methodology below is explicitly designed to simulate practical, real-world, user-centric systems and tasks.

With the definitions below, our general purpose is to design a systemic solution that guarantees all Tasks can be completed as specified, for all Systems. For our purposes, it is important to control our assumptions about the capabilities of individual Machines; we are trying to make this as hard as reasonably possible for ourselves:

• You may assume that if a constraint is specified, a Device meeting that constraint exists in the System (with unspecified location).
• If a Device’s location is ambiguous, or it is ambiguous which extant Device is specified, you may not artificially localize it, unless that location is provably true given the System or Task description.
• You may not assume any hardware Device exists that is not specified or necessarily implied, and you may not specify or assume the addition, modification, or relocation of hardware Devices.
• You may add software constraints to the system, so long as each piece of software added has natural constraints of its own, and those constraints are fulfilled by the System as described (and as amended in your description).
• Insofar as software is specified to exist but not implied to ever run, you may not presume that any requirements implicit or specified by the description exist in the System.

Note that the descriptions in this document are made broader than I will use in the examples below.

Term Definitions:

“Task” - A set of one or more Actions. You may take as granted that the Task’s description of a System is correct, even if it amends the System definition, but only for the duration of that Task.

“Action” - A one-time or ongoing event described in natural language, using established or plausible Actors as constraints.

“Actor” - A Device or Machine that has been established in the System or Task description, or which can be assumed for the rest of the Task description. If the Task’s description of an Actor directly implies Devices or capabilities not yet established, you may assume they exist.

“Constraints” or “Requirements” - When we define a Task or Action, it will implicitly or explicitly stipulate one or more Devices and/or Machines. These stipulations may be specific or general. Failing to meet any constraint specified in a Task marks a solution as invalid.  "Natural" constraints exist for all Actors of that class (eg, Machines or Applications).

“System” - A set of one or more Machines connected by a Backbone. If the System is specified as an actor, it refers to an otherwise unspecified Machine or Device within that system.

“Backbone” - A logical network connecting all Machines in a System. The Backbone’s architecture and properties are arbitrary and can be heterogeneous, so long as the ability to map the network and exchange messages between any two nodes is preserved. Access to the Backbone does not constitute Internet access.

“Machine” - A hardware or virtual computer consisting of one natural Compute Device, and zero or more accessory Devices, described in natural language. No accessory Devices may be presumed except as specified or implied its description or the Task description.  If the description of a Machine implies it is used a specific way (eg Desktop), you may assume any required devices.  Any Machine may be used as an Actor.

“Device” - An arbitrary hardware or software component of a computing system, described in natural language, with all reasonable capabilities and constraints implied by that description. Each Device is associated with a Machine, and unless specified in an Action, shall not be shared or moved to another Machine.  Any Device may be used as an Actor.

“Compute” - A class of Devices used to run programs; specifically, a general-purpose processor of some capacity, and computer memory of some capacity. Insofar as two Compute resources are incompatible, they will be listed as separate Device types. In the case of a Natural Compute requirement (implicit to the definition of a Machine, or any Device that otherwise has Compute), boot media and an operating system are also implied. In all other cases, compute does not imply Storage.

“Storage” - A class of Devices used to persistently store data. Storage may be implied by the existence of data with no given source, or by a Machine being the target of a data write action. Readable storage does not imply writable storage, and vice versa.

“Internet” - Access to any Machine or System outside the System as defined. Internet access shall be classified as a Device, and therefore shall be the property of a Machine, not the Backbone.

“Application” - Software Device in either a Runnable (in Storage) or Running (in Compute) state, described in natural language, which may have zero or more Device Constraints explicitly defined or implicit to the description. Software specified to exist in a certain place does not mean it is running there, except insofar as it performs an active Action. Software specified to be running in a certain place does not mean it is stored there unless no other source is given. The word Application does not imply user interaction (and therefore does not imply the existence of HID devices) unless specified or implied by the application description or an Action.

Goal

Our current methodologies are generally machine-focused, especially our definition of Applications; our assumptions are that all required Devices exist at the Machine level. MAD/ADA is a system-focused application model; it assumes that all required Devices exist at the System level, without specifying that they must be colocated on a single Machine.

The minimum criteria for a solution is that you feel comfortable explaining to software developers of average capability, how they should solve any given task on any given system. An elegant solution will be one in which the same description works for most or all Tasks and Systems.

Under MAD/SAD ADA, for example, for each required Device, an application developer shall look up the standard API as listed in the SAD, and then create an Application Fragment that contains all the logic in their Application that relates specifically to that Device. Application glue code and non-Device logic shall be relegated to one or more Application Fragments that have no other requirements beside Compute. All Fragments shall be written at once, with each Fragment in a separate code namespace. This should suffice for an Application to interact with multiple Devices specified to exist at the System level, without reference to their position in either absolute or relative terms. If you must create a Device Driver, this shall be an Application consisting of a single Fragment dependent on either a low-level device driver, or one or more low-level connectivity device drivers (USB, PCIE, I2C, etc), which provides a public API, which shall be published in the SAD.

Because this description should work for all Systems and Tasks, I will categorize this as an elegant solution to the problem of Generalized Distributed Computing.

The remainder of this post consists of example Systems and Tasks, with which you may test yourself and the ADA. If I’ve made some dumb mistake and you find a flaw in my definitions, feel free to let me know; I haven’t proven that the ADA works, nor that the above problem description is sound, even though I believe in them both, and the problem itself is more important than my ego.  You can also feel free to suggest additional Examples of any complexity.

Examples

Consider the following example Systems, and Tasks to be performed on those Systems.

• A System consisting of five Machines, described as “Desktop”, “Laptop”, “Server”, “Smart TV”, and “Phone”

  • Using an application stored on the Phone, and data stored on the Laptop, process that data on the Server and display a summary on the Smart TV.
  • Using the remote for the Smart TV, send a message to an Application on the Server, to play an internet radio stream on a speaker.
  • Use a camera, display, microphone, and speaker to perform a video call over the internet.
  • Using a text editor application stored on the Desktop, edit files stored on the Phone using the Laptop as an interface.
  • Using files and an application stored on the Server, provide a generic media player whose media library and state is synchronized across the Desktop, Smart TV, Laptop, and Phone.

• A System consisting of five Machines, described as “Phone”, “Work Laptop”, “Home server”, “Work Server”, and “Rented VPS”

  • Using an application stored on the Home Server, edit documents stored on the Work Server using the Work Laptop as an interface.
  • Use the Rented VPS to pass video stored on the Home Server through a filter, and stream the resulting video to the Phone.
  • Using an application on the Work Server, periodically send the location of the Work Laptop to an application on the Work Server. Using an application on the Home Server, periodically send the location of the Work Laptop and Phone to an application on the Rented VPS. If that application on the Rented VPS detects an anomaly, send an alert to the Phone.

• A System consisting of an “AMD64 Desktop”, a “Quantum Processor Host”, a “Scanning Electron Microscope”, and an “Animatronic Monkey”

  • Follow commands send to a web service hosted on the Desktop to control the Scanning Electron Microscope. The output of the Microscope shall be processed by the Quantum Processor for signs of unusual signals. When processing is finished, the animatronic monkey will nod or shake its head to indicate the presence or absence of unusual activity, in view of a webcamera attached to the desktop, which is being live-streamed.
  • Following a program stored on the Desktop, a sample shall be studied with the Microscope and processed on the Quantum Processor, with results stored on the Desktop. Whenever a set of results arrives, the animatronic monkey will perform an interpretive dance based on those results, according to a program stored on the Desktop.
  • A machine learning algorithm loaded onto the Desktop and using the quantum processor for random number generation shall attempt to use the animatronic monkey to safely operate the scanning electron microscope.

• A System consisting of one thousand general purpose Machines, each with no storage, and ten data storage Machines, one of which has access to an external network.

  • The ten data storage machines, shall farm data processing requests, all using the same program but different parameters, to the thousand general purpose Machines. Each general purpose machine has a three-color LED attached, which shall be Red while idle, Yellow while in a setup process, or Green while processing.
  • The thousand general purpose machines each have an attached speaker, and the ten storage machines each have an attached microphone. The general purpose machines each run machine learning algorithms, with the goal of audibly recreating the works of Shakespeare. The storage machines shall judge the success or failure of the endeavor, and will periodically halt this process, and use the results to train a future generation of the algorithm, which is then deployed so that the process may repeat. Successful completion of this Task implies that the loop functions normally for an arbitrary number of iterations, and does not imply that the machine algorithm ever succeeds.

• A System consisting of a “Car Dashboard”, a “Phone”, a “Smart TV”, a “Home Desktop”, and a “Work Desktop”.

  • While the Phone is within range of the Car Dashboard, use the Dashboard as an interface to the Phone, and route all phone and video calls through it.
  • While the Phone is within range of the Smart TV, make the TV available as a secondary display for the Phone. If an incoming call or text arrives while the TV is in use as a secondary display, place a notification there.
  • When the Phone is within range of the Home Desktop or Work Desktop, make the Phone’s apps and data available to run or be accessed by the appropriate Desktop.
  • When the Phone is within range of the Home Desktop or the Work Desktop, make a selection of apps and data stored on the Phone available; the selection of apps and data shall be different for each Machine.
  • When the Phone is within range of the Home Desktop or Work Desktop, make the Phone available as a secondary touchscreen display for the Desktop.
  • Make the Smart TV available as a secondary display for the Home Desktop. Applications available on the Desktop or Phone (if it is in range) can take control over the Smart TV temporarily.
  • While the Work Desktop is in use, silence notifications on the Phone and disable usage of the Smart TV.