Article 3. Part I.
Temporal Paradigm:
Formalization
Abstract
The fusion of biological and digital memory sooner or later will collide with a contradiction: the information in our memory is not data, but experience.
In the context of this work the difference between data and experience is defined by two fundamental aspects:
• Formalization — experience does not exist outside of time, and therefore must be formalized through time;
• Observer — experience does not exist without an Observer.
The first part will focus on Formalization.
Temporal Structure Annotation.
Preprint
In Progress
Spatial Paradigm
The Spatial Paradigm formalizes information, resulting in such an abstraction as data. If we consider formalization at the discrete level, the syntax contains only one language object — the binary digit 1. The use of a single object does not make the formalization spatial, but it becomes so because of the semantics, i.e., the conscious logic that operates on that object [1].
This logic is clearly reflected in the interface (Fig.1), where information (territory) is represented as if it were in some space (map) and can be interacted with as a physical object [2]. Hence the plethora of spatial metaphors, both for objects [3] and actions [4]. This is clear evidence that information is represented spatially because it is primarily understood spatially.
a. macOS.
b. Windows.
Fig.1. Examples of Spatial Representation of Information.
The language of Spatial Paradigm with one symbol and spatial logic allows us to store information that can be accessed through an interface. Anything described in such a language becomes spatial structures, which are summarized here in one general class — Digital Memory.
Because nearly everything that exists in physical space can be stored in the digital memory, theories such as Digital Physics [5] and the Simulation Hypothesis [6] are arising, reflecting the idea that digital memory claims to be the most complete formalization of physical reality.
However, there is no data in our own memory, so everything can be represented spatially except for the "Zero Point", in which this space originates, i.e., in the Observer's point of view (Temporal Space).
At the discrete level, bits of information can be represented as an electrochemical stimulus i.e., as a signal at the input/output of an artificial neuron, but further progress in formalizing biological memory, still "leaves open the question whether it will advance our understanding of the causal links between brain structure and function" [7].
Although biological memory may represent a structure, such a structure eludes spatial formalization. Otherwise, we would already have a proven model of biological memory explaining thinking and consciousness. Therefore, if we stipulate that there is nothing beyond space-time, and biological memory cannot be formalized spatially, then it must be a temporal structure. Structures that can also be reduced syntactically to a single object, but resist spatial formalization, we will generalize into another general class — Biological Memory.
In sum (Fig.2):
• Digital memory — one entity operated by spatial logic and formalizing information into data; spatial structure;
• Biological memory — one entity operated by non-spatial logic and formalizing information into experience; temporal structure.
Fig.2. Digital and Biological Memory.
Temporal Formalization
Temporal structures constitute the very reality within which there is Consciousness engaged in formalization. However, the throughput of Consciousness will always be orders of magnitude lower than that of the temporal structure as a whole [8]. Perhaps the fact that temporal structures remain spatially non-formalized suggests that Consciousness is essentially a local domain within non-local memory. Being local, it can only operate with a language that has a spatial logic that cannot be applied to formalize temporal structure.
However, anything that is formalized by Consciousness becomes local only by definition (of Consciousness), not by nature. Consciousness certainly "contains" abstractions, recognized at a level that is beyond its limits, which then, for reasons that lie outside of Consciousness, can become available for formalization. Such abstractions will in turn be captured in their spatial logic, or state that such logic is violated [9].
To solve the problem of "formalizing temporal structures" we need a non-spacial formalization, and to exclude the possibility that it might carry a fundamentally embedded spatial logic, we have no choice but to make such a formalization inaccessible to Consciousness directly.
The possibility of such "inaccessible formalization" is based on the fact that Consciousness is not isolated and contains abstractions that, although not formalized, but are not absent for this reason. For example, the concept of tacit experience [10], means literally that some knowledge cannot be communicated by language, but nevertheless language is sufficient to formalize the very concept of tacit experience. Such "understanding without formalization" is secured due to the fact that the sensory capabilities of all people are practically the same, i.e., we understand each other without words because we can "feel the same thing".
Time Abstraction
The nature of temporal structure is expressed by such an abstraction as time, and the perception of time is obviously inherent in all humans in one way or another [11]. And although the fundamental definition of time remains non-formalized, Consciousness has created many operational definitions that contain, though not time as such, but properties of our perception of time.
Thus, temporal formalization began in the "Image of Time", with a small selection of tools for measuring (visualizing) time that reflect the following properties of our perception (Fig.3):
• Cyclicity — reflected in the clock face;
• Continuance — reflected in the timeline;
• Irreversibility — as a common feature of all tools.
Fig.3. Properties of Time in Tools.
And continued in "Temporal Geometry", where these properties were formalized geometrically and considered as a syntax, which at the discrete level also contains just one language object – Event-Process.
Inaccessible Language
"Inaccessibility" does not mean that we are deprived of a view of formalization in general. Perhaps syntax can be contained in the locality of Consciousness and only semantics remains beyond its limits. As an example, we can imagine a “language” that has meaning only when all the words exist simultaneously. Or one could say that it has only one "word" the full meaning of which exists only in the present and is determined by the entirety of the Observer's sensorics. It is impossible to make a clear distinction between writing and reading here, as they occur simultaneously (Article 3. Part II).
Nevertheless, if we consider reading in isolation the "word" loses its integrity and is read (defined) by the Dimensionality of Recognition (Article 3. Part II). Consciousness in such a picture operates with a limited dimension, which, firstly, is limited by the length of the "word" in time, and secondly, by the capacity of sensorics observable by Consciousness.
Sensorics, being incomplete, recognizes only part of the symbols, which will result in gaps in the “word”. The gaps, will divide the “word” into groups of symbols, which considered altogether can be defined as a “sentence”. The meaning of this "sentence" is determined by the activity of the incomplete sensorics, during the act of observation (Fig.4).
Fig.4. Partial Recognition based on Dimensionality.
Obviously, such a language in its entirety cannot be consciously perceived. A person cannot be constantly aware of each of their neurons and each of their nerves, or simply be in Consciousness all the time [12]. However, for digital memory, this seems to be no more than a technical challenge. As stated in the "Temporal Geometry" manuscript:
An Artificial Observer can have much larger sensory capabilities (than a human), be simultaneously in all deployed (accessible) dimensions, and observe one or many processes in all dimensions simultaneously.
Fig.5. Recording a Simple Stimulus in Dimension 2T (side view).
Fig. 5 shows the Signature of a single stimulus in the 2T dimension with a duration of 3 seconds. Such a Signature (in the presented side projection) looks similar to a handwritten text, but even if the same stimulus is re-recorded (if we imagine that absolutely identical processes exist in nature), the configuration of the re-recording will be different. Even if the Observer has only one sensory channel, it still shifts in time along the Temporal Axis. Thus, while in natural language the same sentence can be repeated any number of times without changing its meaning, this is not the case with temporal formalization (simply because you can go back to the same point in space, but not in time; Irreversibility). The meaning of such a sentence will always depend on the current specific Dimensionality of Recognition.
Translation
Here it is much more important to solve not the problem of operating the temporal language, but the problem of translation from it to the natural language. At present, digital memory is already capable of operating with natural language within the framework of Large Language Models (LLM) [13]. We can say that it is already capable of translating from human language to human language.
LLM can be seen as a potential mediator for translation from temporal language to natural language. This is the approach already applied in Computational biology and bioinformatics (Fig.6) where language models are used to “interpret protein sequences as sentences and their constituent – amino acids – as single words” [14].
Fig.6. NVIDIA Clara annotates the Data Analyzed by a Sequencer.
However, unlike proteins, translating a temporal language leaves a difficult problem of verifiability since one of the languages is inaccessible to Consciousness and appeals to experience, which is largely not only tacit, but more often quite unique to each Observer.
An approach to this problem might be to limit the sensorics of digital memory to the scale of biological memory or to a limitation characteristic of short-term memory [15]. But most likely, digital and biological memory will never be able to bilaterally understand each other “without words” because of the impossibility of a common tacit experience. Therefore, the probability that this will be a translation from an inaccessible language into a language that only pretends to be human remains significant.
Approaching the formalization of temporal structures as a translation problem contains the most difficult questions not only on the technical but also on the ethical side.
So far, the research is at the level of syntax and semiotics is still quite far away. Nevertheless, having this time, it is already worth using it to determine an ethics of sharing experience, which takes the clearest outlines if we consider digital memory (under certain conditions) as an Artificial Observer. This is the subject of the following Article 3. Part II.
"
Resources:
[1↑] Kowalski, R. (2011). Introduction: Computational Logic and the language of thought. In Computational logic and human thinking how to be artificially intelligent (pp. 2–5). essay, Cambridge University Press.
[2↑] Korzybski, A. (1948). 3 On Structure. In Selections from science and sanity: An introduction to non-Aristotelian Systems and general semantics (pp. 31–39). essay, International Non-Aristotelian Library Pub. Co.
[3↑] Ware, C. (2004). 8. Visual Objects and Data Objects. In Information visualization: Perception for design (pp. 293–325). essay, Elsevier/Morgan Kaufman.
[4↑] Ware, C. (2004). 10. Interacting with Visualizations. In Information visualization: Perception for design (pp. 293–325). essay, Elsevier/Morgan Kaufman.
[5↑] Wikipedia contributors. (2023, June 20). Digital physics. In Wikipedia, The Free Encyclopedia. Retrieved 18:27, September 5, 2023, from [link]
[6↑] Wikipedia contributors. (2023, September 2). Simulation hypothesis. In Wikipedia, The Free Encyclopedia. Retrieved 18:29, September 5, 2023, from [link]
[7↑] Fan, X., & Markram, H. (2019). A brief history of simulation neuroscience. Frontiers in Neuroinformatics, 13. [doi]
[8↑] Shapiro, L., & Spaulding, S. (2021, June 25). Embodied cognition. Stanford Encyclopedia of Philosophy. [link]
[9↑] Musser, G. (2016). Spooky Action at a distance: The phenomenon that reimagines space and time - and what it means for black holes, the big bang, and theories of everything. Scientific American / Farrar, Straus and Giroux.
[10↑] Polanyi, M. (1967). The tacit dimension. Routledge & Kegan Paul.
[11↑] Núñez, R., Cooperrider, K., Doan, D., & Wassmann, J. (2012). Contours of time: topographic construals of past, present, and future in the Yupno valley of Papua New Guinea. Cognition, 124(1), 25–35. [link]
[12↑] Jaynes, J. (2000). The origin of consciousness in the break-down of the bicameral mind. Houghton Mifflin.
[13↑] Lee, A. (2023, June 5). What are large language models and why are they important?. NVIDIA Blog. [link]
[14↑] Elnaggar, A., Heinzinger, M., Dallago, C., Rehawi, G., Wang, Y., Jones, L., Gibbs, T., Feher, T., Angerer, C., Steinegger, M., Bhowmik, D., & Rost, B. (2022). ProtTrans: Toward understanding the language of life through self-supervised learning. IEEE Transactions on Pattern Analysis and Machine Intelligence, 44(10), 7112–7127. [doi]
Figures:
Figure 4, 5 and the Illustration in the Title were generated using Temporal Geometry Simulation.