"This above all; to thine own self be true."
William Shakespeare (Polonius' advice to his son Laertes), "Hamlet" (1601)
"... great life, if you don't weaken"
Irving Klock, Farmer, Herkimer County, NY (1956)
"Stop! All that can be said has been said!."
Eugene Wigner, communication to Lester Ingber (1965)
After my conference talk, when he took me aside and asked why nuclear forces
were so hard to calculate in detail, I replied that the nuclear force was a
cancellation of kinetic and potential energies, and furthermore the potential
energy itself, at many energies, was a cancellation of repulsive and attractive
forces from different particle-exchange contributions. I then started to give
him some further details, whereupon he put his hand up in air and said the
above.
"Thinking about thinking can make you crazy."
Richard Feynman, Caltech, communication to Lester Ingber (1972)
[http://www.feynmanlectures.caltech.edu/info/
-> Stories -> Lester Ingber]
Physical Studies Institute LLC (PSI) develops projects in areas of expertise
documented in the ingber.com InterNet archive. Terms of use, downloading
policies, and consulting/contracting are discussed in the file
ingber_terms.html
Projects and interests are described in
ingber_projects_brief.pdf
2008
For 40+ years I have replied to queries on various aspects of files, papers, and codes, now many of which are on my website https://www.ingber.com/ , especially on: my Adaptive Simulated Annealing (ASA) and path-integral PATHINT & PATHTREE papers and codes; my Karate books and correspondence; my Statistical Mechanics modeling of Financial Markets (SMFM), Neocortical Interactions (SMNI) and Combat (SMC); and on some earlier work in theoretical nuclear and elementary-particle physics. I welcome correspondence from people interested in these disciplines.
My more recent projects, e.g., Ideas by Statistical Mechanics (ISM) and Real Options for Project Schedules (ROPS), span many disciplines and businesses. I especially welcome correspondence from people interested in these projects.
Climbing up from zero in several disciplines has given me a strong appreciation of various stages of development of projects and people, and of the role of interdisciplinary synergies often required to forge successful projects. These experiences have taught me to value/evaluate the person more than any labels defining chosen disciplines.
I also welcome correspondence from people interested in simply registering their comments, although I may not have time to pursue any lengthy correspondence.
Jul 16
See
Some current papers are
ABSTRACT: Background: Previous papers have developed a statistical mechanics of neocortical interactions (SMNI) fit to short-term memory and EEG data. Adaptive Simulated Annealing (ASA) has been developed to perform fits to such nonlinear stochastic systems. An N-dimensional path-integral algorithm for quantum systems, qPATHINT, has been developed from classical PATHINT. Both fold short-time propagators (distributions or wave functions) over long times. Previous papers applied qPATHINT to two systems, in neocortical interactions and financial options. Objective: In this paper the quantum path-integral for Calcium ions is used to derive a closed-form analytic solution at arbitrary time that is used to calculate interactions with classical-physics SMNI interactions among scales. Using fits of this SMNI model to EEG data, including these effects, will help determine if this is a reasonable approach. Method: Methods of mathematical-physics for optimization and for path integrals in classical and quantum spaces are used for this project. Studies using supercomputer resources tested various dimensions for their scaling limits. In this paper the quantum path-integral is used to derive a closed-form analytic solution at arbitrary time that is used to calculate interactions with classical-physics SMNI interactions among scales. Results: The mathematical-physics and computer parts of the study are successful, in that there is modest improvement of cost/objective functions used to fit EEG data using these models. Conclusion: This project points to directions for more detailed calculations using more EEG data and qPATHINT at each time slice to propagate quantum calcium waves, synchronized with PATHINT propagation of classical SMNI.
ABSTRACT: A path-integral algorithm, PATHINT, used previously for several systems, has been generalized from 1 dimension to N dimensions, and from classical to quantum systems, qPATHINT. Pilot quantum applications have been made to neocortical interactions and financial options in 1 dimension. Future work is planned to extend qPATHINT to multiple dimensions in these systems, and to extend calculations to multiple scales of interaction between classical events and expectations over quantum processes. Each of the two systems considered contribute insight into applications of qPATHINT to the other system, leading to new algorithms presenting time-dependent propagation of interacting quantum and classical scales.
ABSTRACT: The author previously developed a numerical multivariate path-integral algorithm, PATHINT, which has been applied to several classical physics systems, including statistical mechanics of neocortical interactions, options in financial markets, and other nonlinear systems including chaotic systems. A new quantum version, qPATHINT, has the ability to take into account nonlinear and time-dependent modifications of an evolving system. qPATHINT is shown to be useful to study some aspects of serial changes to systems. Applications to options on quantum money and blockchains in financial markets are discussed.
ABSTRACT: The author previously developed a numerical multivariate path-integral algorithm, PATHINT, which has been applied to several classical physics systems, including statistical mechanics of neocortical interactions, options in financial markets, and other nonlinear systems including chaotic systems. A new quantum version, qPATHINT, has the ability to take into account nonlinear and time-dependent modifications of an evolving system. qPATHINT is shown to be useful to study some aspects of serial changes to systems. Applications ranging from regenerative $\mathrm{Ca}^{2+}$ waves in neuroscience to options on blockchains in financial markets are discussed.
ABSTRACT: The author previously developed a generalization of a binomial tree algorithm, PATHTREE, to develop options pricing for multiplicative-noise models possessing quite generally time dependent and nonlinear means and variances. This code is generalized here for complex variable spaces, to produce qPATHTREE useful for quantum systems. As highlighted in this paper, a quantum version, qPATHTREE, has the ability to take into account time dependent modifications of an evolving system. qPATHTREE is shown to be useful to study some aspects of serial changes to systems. Similarly, another path-integral code, PATHINT, used for several previous systems is being developed into qPATHINT. An example is given for a free particle, and it is explained when an n-tree generalization of qPATHTREE beyond the binomial tree is required for such systems, similar to code developed for qPATHINT. Potential applications in neuroscience and financial markets are discussed.
October 2016
FREE WILL
Free Will in the context of my current project, using Statistical Mechanics of Neocortical Interactions (SMNI) under The Extreme Science and Engineering Discovery Environment (XSEDE.org)
From Section 3, L. Ingber, "Evolution of regenerative Ca-ion wave-packet in neuronal-firing fields: Quantum path-integral with serial shocks," Report 2017:QPIS, Physical Studies Institute, Ashland, OR, (2017). [ URL https://www.ingber.com/path17_quantum_pathint_shocks.pdf ]
In addition to the intrinsic interest of researching short-term memory (STM) and multiple scales of neocortical interactions via electroencephalography (EEG) data, there is interest in researching possible quantum influences on highly synchronous neuronal firings relevant to STM to understand possible connections to "Free Will" (FW).
As pointed out in some recent papers (Ingber, 2016a,b), if neuroscience ever establishes experimental feedback from quantum-level processes of tripartite synaptic interactions with large-scale synchronous neuronal firings, that are now recognized as being highly correlated with STM and states of attention, then FW may yet be established using the quantum "Free Will Theorem" (FWT) (Conway and Kochen, 2006, 2009).
Basically, this means that a Ca2+ quantum wave-packet may generate a state proven to have not previously existed. In the context of the basic premise of this paper, this state may be influential in a large-scale pattern of synchronous neuronal firings, thereby rendering this pattern as a truly new pattern not having previously existed. The FWT shows that this pattern, considered as a measurement of the Ca2+ quantum wave-packet, surprisingly is correctly identified as itself being a new decision not solely based on previous (deterministic) decisions, even under reasonably stochastic experimental conditions.
Only recently has the core Statistical Mechanics of Neocortical Interactions (SMNI) hypothesis since circa 1980 (Ingber, 1981, 1982, 1983), that highly synchronous patterns of neuronal firings in fact process high-level information, been verified experimentally (Asher, 2012; Salazar et al., 2012).
Clearly, even in the above context, for most people most of the time, internal and external events affecting neural probabilistic patterns of attention give rise to quite practical reasonable FW. However, there also may be some Science that establishes a truly precise FW.
REFERENCES
Asher, 2012. Asher, J., "Brain's code for visual working memory deciphered in monkeys NIH-funded study," NIH Press Release, NIH, Bethesda, MD (2012). URL http://www.nimh.nih.gov/news/science-news/2012/in-sync- brain-waves-hold-memory-of-objects-just-seen.shtml.
Conway & Kochen, 2006. Conway, J. & Kochen, S., "The free will theorem," arXiv:quant- ph/0604079 [quant-ph], pp. 1-31, Princeton U, Princeton, NJ (2006).
Conway & Kochen, 2009. Conway, J. & Kochen, S., "The strong free will theorem," Notices of the American Mathematical Society 56(2), pp. 226-232 (2009).
Ingber, 1981. Ingber, L., "Towards a unified brain theory," Journal Social Biological Structures 4, pp. 211-224 (1981). URL https://www.ingber.com/smni81_unified.pdf.
Ingber, 1982. Ingber, L., "Statistical mechanics of neocortical interactions. I. Basic formulation," Physica D 5, pp. 83-107 (1982). URL https://www.ingber.com/smni82_basic.pdf.
Ingber, 1983. Ingber, L., "Statistical mechanics of neocortical interactions. Dynamics of synaptic modification," Physical Review A 28, pp. 395-416 (1983). URL https://www.ingber.com/smni83_dynamics.pdf.
Ingber, 2016a. Ingber, L., "Path-integral quantum PATHTREE and PATHINT algorithms," International Journal of Innovative Research in Information Security 3(5), pp. 1-15 (2016a). URL https://www.ingber.com/path16_quantum_path.pdf and https://dx.doi.org/10.17632/xspkr8rvks.1.
Ingber, 2016b. Ingber, L., "Statistical mechanics of neocortical interactions: Large-scale EEG influences on molecular processes," Journal of Theoretical Biology 395, pp. 144-152 (2016b). URL https://www.ingber.com/smni16_large-scale_molecular.pdf and https://dx.doi.org/10.1016/j.jtbi.2016.02.003.
Salazar et al, 2012. Salazar, R.F., Dotson, N.M., Bressler, S.L. & Gray, C.M., "Content-specific fronto-parietal synchronization during visual working memory," Science 338(6110), pp. 1097-1100 (2012). URL https://dx.doi.org/10.1126/science.1224000.
July 2009
ABSTRACT: Columnar firings of neocortex, modeled by a statistical mechanics of neocortical interactions (SMNI), are investigated for conditions of oscillatory processing at frequencies consistent with observed electroencephalography (EEG). A strong inference is drawn that physiological states of columnar activity receptive to selective attention support oscillatory processing in observed frequency ranges. Direct calculations of the Euler-Lagrange (EL) equations which are derived from functional variation of the SMNI probability distribution, giving most likely states of the system, are performed for three prototypical Cases, dominate excitatory columnar firings, dominate inhibitory columnar firings, and in-between balanced columnar firings, with and without a Centering mechanism (CM) (based on observed changes in stochastic background of presynaptic interactions) which pulls more stable states into the physical firings ranges. Only states with the CM exhibit robust support for these oscillatory states. These calculations are repeated for the visual neocortex, which has twice as many neurons/minicolumn as other neocortical regions. These calculations argue that robust columnar support for common EEG activity requires the same columnar presynaptic parameter necessary for ideal short-term memory (STM). It is demonstrated at this columnar scale, that both shifts in local columnar presynaptic background as well as local or global regional oscillatory interactions can effect or be affected by attractors that have detailed experimental support to be considered states of STM. Including the CM with other proposed mechanisms for columnar-glial interactions and for glial-presynaptic background interactions, a path for future investigations is outlined to test for quantum interactions, enhanced by magnetic fields from columnar EEG, that directly support cerebral STM and computation by controlling presynaptic noise. This interplay can provide mechanisms for information processing and computation in mammalian neocortex.
This paper demonstrates by explicit calculations that short-term memory (STM) and EEG can indeed be correlated. At least according to some reviewers, this seems not to have been demonstrated previously. This paper shows that the previous SMNI models which calculate many features measured as STM also support EEG at columnar scales. To put this into some perspective, many neuroscientists believe that global regional activity supports EEG wave-like oscillatory observations, by solving wave equations with hemisphere boundary conditions with spherical eigenfunctions that detail the frequencies of EEG. In this columnar study, wave-type equations are derived via nonlinear EL equations from SMNI probability distributions, and these are explicitly numerically solved to demonstrate that observed EEG frequencies are supported under the same SMNI conditions that support STM.
The next study in smni10_multiple_scales.pdf includes definitive calculations using PATHINT to evolve multivariate probability distributions of firing states.
2010
ABSTRACT: The dynamic behavior of scalp potentials (EEG) is apparently due to some combination of global and local processes with important top-down and bottom-up interactions across spatial scales. In treating global mechanisms, we stress the importance of myelinated axon propagation delays and periodic boundary conditions in the cortical-white matter system, which is topologically close to a spherical shell. By contrast, the proposed local mechanisms are multiscale interactions between cortical columns via short-ranged non-myelinated fibers. A mechanical model consisting of a stretched string with attached nonlinear springs demonstrates the general idea. The string produces standing waves analogous to large-scale coherence EEG observed in some brain states. The attached springs are analogous to the smaller (mesoscopic) scale columnar dynamics. Generally, we expect string displacement and EEG at all scales to result from both global and local phenomena. A statistical mechanics of neocortical interactions (SMNI) calculates oscillatory behavior consistent with typical EEG, within columns, between neighboring columns via short-ranged non-myelinated fibers, across cortical regions via myelinated fibers, and also derive a string equation consistent with the global EEG model.
This paper includes PATHINT evolution of probability distributions of columnar activity with explicit oscillatory firings, and integration of such mesoscopic processes with global brain EEG activity.
May 11
The SMNI project has been developed in a series of papers since 1981. Current work includes neuron-astrocyte interactions, bringing more scales of interaction into SMNI STM. A project that will be further developed into two invited papers is described in a report:
Nov 11
A full review of SMNI, including its mathematical outline, in the context of STM is in:
Jun 12
More research on a "smoking gun" for explicit top-down neocortical mechanisms that directly drive bottom-up processes that describe memory, attention, etc., is given in several papers, e.g.:
Feb 16
ABSTRACT: Calculations further support the premise that large-scale synchronous firings of neurons may affect molecular processes. The context is scalp electroencephalography (EEG) during short-term memory (STM) tasks. The mechanism considered is $\mathbf{\Pi} = \mathbf{p} + q \mathbf{A}$ (SI units) coupling, where $\mathbf{p}$ is the momenta of free $\mathrm{Ca}^{2+}$ waves $q$ the charge of $\mathrm{Ca}^{2+}$ in units of the electron charge, and $\mathbf{A}$ the magnetic vector potential of current $\mathbf{I}$ from neuronal minicolumnar firings considered as wires, giving rise to EEG. Data has processed using multiple graphs to identify sections of data to which spline-Laplacian transformations are applied, to fit the statistical mechanics of neocortical interactions (SMNI) model to EEG data, sensitive to synaptic interactions subject to modification by $\mathrm{Ca}^{2+}$ waves.
Jul 06
This project is described in a report:
ABSTRACT: Ideas by Statistical Mechanics (ISM) is a generic program to model evolution and propagation of ideas/patterns throughout populations subjected to endogenous and exogenous interactions. The program is based on the author's work in Statistical Mechanics of Neocortical Interactions (SMNI), and uses the author's Adaptive Simulated Annealing (ASA) code for optimizations of training sets, as well as for importance-sampling to apply the author's copula financial risk-management codes, Trading in Risk Dimensions (TRD), for assessments of risk and uncertainty. This product can be used for decision support for projects ranging from diplomatic, information, military, and economic (DIME) factors of propagation/evolution of ideas, to commercial sales, trading indicators across sectors of financial markets, advertising and political campaigns, etc.
It seems appropriate to base an approach for propagation of ideas on the only system so far demonstrated to develop and nurture ideas, i.e., the neocortical brain. A statistical mechanical model of neocortical interactions, developed by the author and tested successfully in describing short-term memory and EEG indicators, is the proposed model. ISM develops subsets of macrocolumnar activity of multivariate stochastic descriptions of defined populations, with macrocolumns defined by their local parameters within specific regions and with parameterized endogenous inter-regional and exogenous external connectivities. Parameters with a given subset of macrocolumns will be fit using ASA to patterns representing ideas. Parameters of external and inter-regional interactions will be determined that promote or inhibit the spread of these ideas. Tools of financial risk management, developed by the author to process correlated multivariate systems with differing non-Gaussian distributions using modern copula analysis, importance-sampled using ASA, will enable bona fide correlations and uncertainties of success and failure to be calculated. Marginal distributions will be evolved to determine their expected duration and stability using algorithms developed by the author, i.e., PATHTREE and PATHINT codes.
Dec 03
Oct 04
Jan 06
Apr 06
Previous work, mostly published, developed two-shell recursive trading systems. An inner-shell of Canonical Momenta Indicators (CMI) is adaptively fit to incoming market data. A parameterized trading-rule outer-shell uses the global optimization code Adaptive Simulated Annealing (ASA) to fit the trading system to historical data. A simple fitting algorithm, usually not requiring ASA, is used for the inner-shell fit.
An additional risk-management middle-shell has been added to create a three-shell recursive optimization/sampling/fitting algorithm. Portfolio-level distributions of copula-transformed multivariate distributions (with constituent markets possessing different marginal distributions in returns space) are generated by Monte Carlo samplings. ASA is used to importance-sample weightings of these markets. There are many publications in the academic and commercial literature on similar treatments of markets. The Buddha is in the details :).
TRD processes Training and Testing trading systems on historical data, and consistently interacts with RealTime trading platforms -- all at minute resolutions. Faster or slower resolutions can be developed using the present structure of TRD. The code is written in vanilla C, and runs across platforms such as XP/Cygwin, SPARC/Solaris, i386/FreeBSD, i386/NetBSD, etc. TRD can be run as an independent executable or called as a DLL. Some more detail is given in
To illustrate how TRD can robustly and flexibly interact with various trading platforms, I have developed a working interface with TradeStation and have outlined an interface with Fidelity's Wealth-Lab.
Beware Greeks Bearing Gifts: Mathematical models are only as good -- at best -- as the data supporting them. It may not make sense applying models based on historical data to current time. It is important to stress-test models using random data generated in large windows around historical data.
The PSI basic concepts, architecture, and TRD code described above can be used to further develop PSI's published approaches to projects in other disciplines, as well as to generate other projects.
These concepts can be applied to developing tools for decision-makers of companies and government agencies, assembling functions, departments, processes, etc., tailored to specific client requirements, into an overall "portfolio" from which top-level measures of performance are developed with associated measures of risk, and with audit trails back to the member constituents. For example, marginal distributions can be formulated based on many kinds of sets of data. A report describes how options (in the sense of financial options) and risk management can be developed for project schedules:
Adaptive Simulated Annealing (ASA) and path-integral techniques, e.g., PATHINT and PATHTREE, have been published, demonstrating their utility in statistical mechanical studies in finance, neuroscience, combat analyses, neuroscience, and other selected nonlinear multivariate systems. PATHTREE has been used extensively to price financial options.
PATHTREE can be be generalized, from folding forward in time a wide class of nonlinear stochastic 1-dimensional distributions, to n-dimensional distributions in an n-PATHTREE code. PATHINT becomes extremely computationally intensive with just 3 variables, but n-PATHTREE likely can be much quicker with the same extreme accuracy for higher dimensional systems. A matrix formulation of binomial trees could be the focus for developing n-PATHTREE. This algorithm can be used to fit the shape of distributions to data, providing a robust bottom-up approach to "curve-fitting" systematics of empirical data.
These tools also are being applied to price complex projects as financial options with alternative schedules and strategies. PATHTREE processes real-world options, including nonlinear distributions and time-dependent starting and stopping of sub-projects, with parameters of shapes of distributions fit using ASA to optimize cost and duration of sub-projects.
1997
High-quality EEG data has been used to perform ASA optimization of "canonical momenta" indicators (CMI). This work demonstrates how the theory of statistical mechanics of neocortical interactions (SMNI) can describe individuals' macroscopic brain function as measured by raw EEG.
The first paper giving a detailed calculation of CMI in EEG was
ABSTRACT: A series of papers has developed a statistical mechanics of neocortical interactions (SMNI), deriving aggregate behavior of experimentally observed columns of neurons from statistical electrical-chemical properties of synaptic interactions. While not useful to yield insights at the single neuron level, SMNI has demonstrated its capability in describing large-scale properties of short-term memory and electroencephalographic (EEG) systematics. The necessity of including nonlinear and stochastic structures in this development has been stressed. Sets of EEG and evoked potential data were fit, collected to investigate genetic predispositions to alcoholism and to extract brain "signatures" of short-term memory. Adaptive Simulated Annealing (ASA), a global optimization algorithm, was used to perform maximum likelihood fits of Lagrangians defined by path integrals of multivariate conditional probabilities. Canonical momenta indicators (CMI) are thereby derived for individual's EEG data. The CMI give better signal recognition than the raw data, and can be used to advantage as correlates of behavioral states. These results give strong quantitative support for an accurate intuitive picture, portraying neocortical interactions as having common algebraic or physics mechanisms that scale across quite disparate spatial scales and functional or behavioral phenomena, i.e., describing interactions among neurons, columns of neurons, and regional masses of neurons.
A follow-up study, including testing the CMI on out-of-sample data, is in
Additional results (tables of ASA-fitted parameters and 60 files containing 240 PostScript graphs) are contained in smni97_eeg_cmi.tar.gz
Some background and results in lecture-plate form are given in
Dec 06
Some of the algorithms used in the ISM project (above) are used in
An approach to developing a quasi-automated trading system, based on a theory of a statistical mechanics of financial markets (SMFM) has been developed in several papers, e.g.,
ABSTRACT: A paradigm of statistical mechanics of financial markets (SMFM) is fit to multivariate financial markets using Adaptive Simulated Annealing (ASA), a global optimization algorithm, to perform maximum likelihood fits of Lagrangians defined by path integrals of multivariate conditional probabilities. Canonical momenta are thereby derived and used as technical indicators in a recursive ASA optimization process to tune trading rules. These trading rules are then used on out-of-sample data, to demonstrate that they can profit from the SMFM model, to illustrate that these markets are likely not efficient. This methodology can be extended to other systems, e.g., electroencephalography. This approach to complex systems emphasizes the utility of blending an intuitive and powerful mathematical-physics formalism to generate indicators which are used by AI-type rule-based models of management.
The calculations contained in
A brief and less technical discussion of this approach and of ASA is given in
A brief discussion and motivation for work in progress, further developing SMFM, is given in
Application of SMFM to developing volatility of volatility in the context of Eurodollar options is given in
ABSTRACT: The Black-Scholes theory of option pricing has been considered for many years as an important but very approximate zeroth-order description of actual market behavior. We generalize the functional form of the diffusion of these systems and also consider multi-factor models including stochastic volatility. We use a previous development of a statistical mechanics of financial markets to model these issues. Daily Eurodollar futures prices and implied volatilities are fit to determine exponents of functional behavior of diffusions using methods of global optimization, Adaptive Simulated Annealing (ASA), to generate tight fits across moving time windows of Eurodollar contracts. These short-time fitted distributions are then developed into long-time distributions using a robust non-Monte Carlo path-integral algorithm, PATHINT, to generate prices and derivatives commonly used by option traders. The results of our study show that there is only a very small change in at-the money option prices for different probability distributions, both for the one-factor and two-factor models. There still are significant differences in risk parameters, partial derivatives, using more sophisticated models, especially for out-of-the-money options.
A very quick and robust algorithm motivated by PATHINT, PATHTREE, has been developed:
ABSTRACT: Motivated by path-integral numerical solutions of diffusion processes, PATHINT, we present a new tree algorithm, PATHTREE, which permits extremely fast accurate computation of probability distributions of a large class of general nonlinear diffusion processes.
Some background and results in lecture-plate form are given in
A theory of statistical mechanics of combat (SMC) is given in
ABSTRACT: The context of nonlinear combat calls for more sophisticated measures of effectiveness. We present a set of tools that can be used as such supplemental indicators, based on stochastic nonlinear multivariate modeling used to benchmark Janus simulation to exercise data from the U.S. Army National Training Center (NTC). As a prototype study, a strong global optimization tool, adaptive simulated annealing (ASA), is used to explicitly fit Janus data, deriving coefficients of relative measures of effectiveness, and developing a sound intuitive graphical decision aid, canonical momentum indicators (CMI), faithful to the sophisticated algebraic model. We argue that these tools will become increasingly important to aid simulation studies of the importance of maneuver in combat in the 21st century.
Some background and results in lecture-plate form are given in
This interdisiplinary project required cooperation from many sources, which was not very easy at all to obtain:
As part of my agreement to develop a new curriculum in simulation within the physics department, I created a methodology to assess training data at NTC, in order to further develop a JANUS simulation based to this real data. There was a lot of resistance: to the use of physics algorithms in this context, to gaining access to NTC to collect data, and to further expand the first successful Army simulation to Navy and Air Force exercises. This required developing teams from DoD and contractors, and having personnel tasked to these projects for testing of the simulations. After completion, the DUSA OR commended our group for contributing a valuable resource to test tactics and weapons systems based on simulations baselined to real-world physics and data.
This work required development of numerical algorithms to fit data to highly nonlinear multivariate data and to evolve the probability distributions developed by the models. The first point was addressed by my creation of Very Fast Re-Annealing (VFSR), to become Adaptive Simulated Annealing (ASA), which is now used world-wide in numerous disciplines for generic importance-sampling optimization of many complex systems. The second point was solved by collaboration with LLNL contractors to my project, which led to my PATHINT code, and in later years to my PATHTREE code.
My work in developing a simulation program was sponsored and rooted in the NPS Physics Department, demonstrating that real- world applications could and should have peer-reviewed research backing and documentation. To do this effectively, I had to gain the respect of Officers at training sites to gather data (which they had not permitted previously), gain the respect of Army and Navy personnel onsite to build simulations based on raw data and physics algorithms (not in their background), work to have external Officers assigned to work with the simulations (e.g., the Army assigned the 101 Airborne), work with contractors on National labs to test the algorithms (I typically included them in the research papers), etc.
This work was also recognized outside of people familiar with this project: On 25 March 1988, after several panels of technical and managerial review, I was officially notified that I was selected as Assistant Director Washington Operations, Joint Tactical C3 Agency (JTC3A), a Senior Executive Service (SES) position, and that I would have to begin this new appointment immediately. However, I had worked for three years to carve out an approach to realistically baseline combat models to field data, had obtained funding for a civilian-military team from the Deputy Under Secretary of the Army for Operations Research, and (correctly) estimated that we needed a few more months to bring this project to some focus, as I did at CAA from February through June 1989 until the end of my contract with NPS.
All this work alienated at least one group of people at NPS, the Operations Research (OR) department, many of whom felt that all this kind of research and contracts belonged within their domain (although they had not pursued this work previously). They also did not understand path integrals, and this bothered them very much, to the point of denying that any such math-physics existed! A fully documented report is available on request.
21 Oct 95
Here is an edited reply to a query on "canonical momenta," a common feature of the above projects, that may be useful to some readers.
: Just out of curiosity. Could you briefly define "canonical momenta
: indicator?" Very few people understand the definition. If that's
: too much time, please tell me where I could find the information.
Yes, it does require some advanced physics to articulate in detail. When dealing with truly nonlinear multivariate systems, the details go beyond a normal PhD physics education, using some advanced calculus developed in the late '70's.
That said, let me try to give a thumbnail sketch:
A stochastic differential equation (sde) like
This sde can be written as a conditional short-time distribution
The momentum is
In this simple one-dim example, g is just the standard deviation, but in more than one dim, g**2 becomes the covariance matrix. It turns out that this is the inverse-metric of the space as well, and it enters into the calculation of the "canonical momenta." Perhaps the easiest way to see this is to look at the third mathematically equivalent representation, the Fokker-Planck partial differential equation (pde) for p:
A very simple but useful text on the physical relevance of such sde and pdf across many physical and biological systems is
There is a nice chapter on "The Principle of Least Action," illustrating how "F = ma" is derived from the Lagrangian in Volume III, Chapter 19 of
A book that shows all the gory glory of the additional complications that must be dealt with when multivariate nonlinear systems are considered is
My work in neuroscience since the late '70's, e.g., in the smni... papers in my archive, first took advantage of these new mathematical physics developments in multivariate nonequilibrium nonlinear statistical mechanics. I then applied these techniques to markets, nuclear physics, and combat analyses, e.g., in my markets..., nuclear..., and combat... papers in my archive, which give more details and references.
I developed this formalism into a practical maximum likelihood numerical tool for fitting parameters in these kinds of systems with the use of VFSR/ASA, e.g., illustrated with the use of some of the asa... papers in my archive. The ASA code is in
Some background and results in lecture-plate form are given in
The long-time evolution of such multivariate systems is accomplished with the use of algorithms such as PATHINT and PATHTREE, e.g., illustrated in the path... papers in my archive. This is required, for example, to calculate many kinds of financial instruments, e.g., bond prices, options, derivatives, etc. The famous Black-Scholes model for options pricing is such an example of a one-variable distribution. People have developed numerical algorithms for each representation, i.e., for the SDE, PDE, and the Lagrangian probability representations. Examples of the use of PATHINT for options pricing are given in https://www.ingber.com/markets00_exp.pdf and https://www.ingber.com/markets00_highres.pdf and the use of PATHTREE is given in markets01_pathtree.pdf as outlined above.
Some background and results in lecture-plate form are given in
In addition to canonical momenta used as a natural coordinate system to study dynamically evolving multivariate systems, their value has several explicit aspects as used in a code I developed, TRD. With respect to my markets96_trading.pdf and markets96_momenta.pdf papers, in addition to canonical momenta being accurate as well as intuitive measures of "flows" to and from evolving steady states, I have made the points:
These ideas were implemented in a realtime tick-resolution trading system, as described in markets01_optim_trading.pdf
Similarly, in EEG work, it is possible to monitor a subset of electrode activity while incorporating information from additional electrodes.
Papers markets84_statmech.pdf, markets91_interest.pdf and smni91_eeg.pdf contain an Appendix giving a compact derivation of the path-integral Lagrangian representation equivalent to the Langevin rate-equation and Fokker-Planck/Schroedinger-type representations for multivariate systems with nonlinear drifts and diffusions.
My experience with nonlinear systems, based on my own research into selected topics in selected disciplines as well as my interactions with many other experts and nonlinear systems, e.g., through my support of ASA, is that nonlinear systems are typically non-typical. It typically requires quite a bit of work understanding the nature of each each system before applying or evolving models and techniques borrowed from other disciplines.
I hope this helps.
Lester
With just dimension, D = 1, a formula for the probability of having any value x(t) at time t+dt given the value of x(t+dt) at time t would be:
If we are dealing with more than one dimension, D, then G become a matrix and f become a vector, e.g., for x and y:
I have developed the above formulation into a numerical procedure for nonlinear multivariate systems that otherwise would be described as
If [G_i_k] is the matrix of the coefficients in the differential rate equations, and [G_ij] is the covariance matrix (the inverse of the metric matrix for this space) in the Lagrangian discussed above,
As the number of dimensions increases, so do the number of parameters to be fit. As the functions F and G become more nonlinear, as they will with increasing D, and as there definitely is "noise" in the data, it becomes increasingly important to use some powerful global optimization code like ASA to fit the data, else the coefficients will just be modeling the "noise."
An additional hard problem arises in forecasting P for times greater than dt in the future. You have to do the "path integral," i.e., integrals over all the x's for each of the intermediate times:
Only if very simple forms for F and G are selected, can these integrals be performed to give a closed expression; otherwise they must be performed numerically. PATHINT and PATHTREE do this for _some_ classes of integrals. However, even for dimension D = 2 the CPU time is large, and for D = 3 or 4, this might be huge. If simple forms are taken for the F's and G's, all the global optimization fits still must be performed, but perhaps you can get by without using something like PATHINT.
A paper on the general utility of this approach to modeling complex systems is given in
ABSTRACT: A modern calculus of multivariate nonlinear multiplicative Gaussian-Markovian systems provides models of many complex systems faithful to their nature, e.g., by not prematurely applying quasi-linear approximations for the sole purpose of easing analysis. To handle these complex algebraic constructs, sophisticated numerical tools have been developed, e.g., methods of adaptive simulated annealing (ASA) global optimization and of path integration (PATHINT). In-depth application to three quite different complex systems have yielded some insights into the benefits to be obtained by application of these algorithms and tools, in statistical mechanical descriptions of neocortex (short-term memory and electroencephalography), financial markets (interest-rate and trading models), and combat analysis (baselining simulations to exercise data).
Jun 10
PATHTREE was developed in response to a problem faced by a trader, who felt that the Black-Scholes (BS) pricing model was too sensitive to the BS distribution. His empirical solution was to distort the scale of input underlying variables, e.g., by factors of thousands (larger than any prices on the open market!?), and then to similarly scale back the output. I thought that using the actual distribution of the data would be best, even if the distribution was quite an odd-shape -- as long as the cumulative distribution was a bona fide distribution. This meant developing an algorithm that, while still delivering all common Greeks used for trading, could process distributions with quite general first and second moments, e.g., nonlinear in the dependent variables and time, etc. The focus was to fit parameters defining such odd shapes to strikes of the options, thereby developing a bottom-up approach to "smiles" (non-linear curves of strikes versus variables like volatility). (The usual top-down approach to smiles is to interpolate/extrapolate output of BS models to the strike data.)
Just about as soon as I articulated the problem, I realized that my past published works in several disciplines, using nonlinear nonequilibrium multivariate Gaussian-Markovian distributions, was relevant. That is, it turns out that even though the first two moments of a distribution might only be accurate to order (dt)^(1/2), the conditional probability distribution is accurate to order (dt)^(3/2). (This requires quite a bit of heavy math to prove, as referenced in my publications.) Accuracy to order (dt) is required to use standard numerical binomial trees to solve the associated options equations, whether written equivalently as differential stochastic equations, Fokker-Planck equations, or path-integral equations, the latter being the representation of the binomial tree algorithm. In a few minutes I modified the binomial-tree code we were using to test this idea, and it worked perfectly!
I then had my team thoroughly test this algorithm, by calculating and plotting all options being used by all traders in the firm, with respect to all variables and parameters being used at the time for their BS codes, e.g., asset price, strike price, time to expiration, risk free rate, cost of carry, volatility, volatility step size, number of time steps, American and European options, and boundary values at far distances (factors of 1/10 to 10) from actual values touched by the present calculations, to ensure proper behavior for extreme calculations. This produced thousand of pages of graphs, which were divided among the team to examine for any odd behavior. We found none. Just to be careful, and email was sent to all traders, telling them that if they started trading any new options or present options within new ranges of any variables, they were to first contact me so I could run their options through these tests which were now part of the code.
As it of course would turn out, someone did not heed the warnings. The trader for whom the new algorithm was developed just decided that reality was not good enough, and he persisted in using scales at thousands of times the actual data. After a while, going along quite well but not telling us of his mangling of the basic codes, he hit a problem in some discontinuity of results among different strikes and he lost a lot of money, which he attributed probably incorrectly to the code per se. When he called in the midst of trading, after a few minutes of reflection, I had my team immediately run his values, and we in fact saw this small discontinuity. I immediately guessed it had to do with the way boundaries were being enforced across the parameterized distribution and the far-away distribution (which he was hitting with his huge scaling of the data). This was an easy immediate fix and the new code was delivered to him in less than half an hour after his call. Of course I was blamed for the error, which was true, except of course the error would never have been used if the trader has used the due diligence he was warned to exercise.
The "moral" of this incident is that (a) "Black Swans" (unknown unknown future events beyond known stochastic models) always are potentially threatening, (b) risk controls must be in place to mitigate such threats, and (c) all players must follow the discipline of such risk controls. We had risk controls in place, but they were not followed when the model being used was used beyond its intended context for which it was tested within broad risk bands of 10-100 (but not 1000's). Mathematical models are just those, and they not only depend on good data, but also on controls to account for errors/deficiencies in the models when they are stretched beyond their original domains. Such abuses also were present on the first decade of this century, leveraging the nightmare of the meltdown of the global financial system.
This new algorithm was eventually published in:
1997
Every worthwhile project I've undertaken -- from Nuclear Physics to Statistical Mechanics (SM) projects to Karate -- each took some years of study to understand a basic set of problems in a given discipline. It just takes a lot of time to get familiar with the current issues of a discipline as viewed by experts, to understand the most important unsolved or poorly solved issues, and to see if new approaches can be applied to create better solutions, without being distracted by focusing on easier different problems that do not address the important issues. It is to be expected that a newcomer to any discipline should be expected to do the "homework" required. I addressed real problems requiring such studies, not only to appreciate the core problems at an expert level, but to forge the necessary tools to solve these problems. In the SM studies, this required creating such tools as ASA, PATHINT and PATHTREE, and many other lesser-known algorithms.
Some tools had to be developed to accommodate particular constraints. Similar to optimizing some complex systems, some projects require constraints additional to simply developing analytic solutions to scientific problems, e.g., including an institution's focus, capabilities of team members, ability of decision makers to work with results, etc. These projects can be quite interesting and challenging, requiring blending mathematical science with social or personnel objectives, forming a larger system to be considered. In such cases, once a reasonably solid intuitive understanding is reached for an approach, analysis should strive to not waiver from this intuition, but to be faithful to the original understanding. Of course, often further analysis uncovers new aspects of a system demanding attention and modification, often requiring problems and solutions to converge only at late stages of solution. This is different from simply giving into a too common temptation for analysts to bend their analysis to suit the ease of solution rather than to deliver projects that solve the original problem.
When my Statistical Mechanics of Neocortical Interactions (SMNI) approach first appeared circa 1978, it was not hard to understand why many people in neuroscience trained either in medicine or abstract mathematics could not readily understand this mathematical-physics approach to the neocortex. Especially at that time, neural networks were much simpler to understand, albeit conveniently glossing over issues of relevance to real neocortex. However, even supporters of this work with extremely solid scientific credentials did not stop a few other reviewers from attempting to cut funding and publication of this work. One early reviewer, a well-known neurologist claimed that the math was fabricated as a smoke screen to cover the results claimed by the algebraic and numerical calculations. Since then, SMNI papers have successfully applied this math-physics to numerically detail properties of EEG and short-term memory.
When my Statistical Mechanics of Financial Markets (SMFM) approach first appeared circa 1982, the paper finally published in 1984 was delayed a couple of years. For example, an editor of a premier economics journal agreed with a reviewer (who stated he had a graduate-student physics background) that such math as used in that paper did not exist and could not be correct. Since then, this math-physics has successfully been applied to derivatives, trading systems and risk-management of portfolios. Success in trading firms I've worked with, albeit a profitable enterprise, most certainly is not documentation of validity of this work, and I have managed to publish some of this work to get genuine peer review.
When my Statistical Mechanics of Combat (SMC) approach first appeared circa 1987, in the context of studies of studies on large-scale government simulations, several operations-research/statistician government analysts stated a similar complaint that they could not find any such math in their previous math or physics textbooks. One OR professor actually sent out letters to Department Heads claiming I was promoting bogus studies. They claimed "path integrals" were only a theoretical abstraction without any foundation for application to anything except quantum mechanics -- pretty crazy complaints!. After the dust settled somewhat, I was thanked by an Asst. Secretary of the Army for my work in promoting this work, which led to use of these simulations in training. Upon request, I may give a URL to a file giving more detail on this bizarre tragicomedy, including intervention required by a Congressman.
Clearly, the overwhelming majority of other reviewers of my work, who endorsed publication, have at the least established that these fewer critics were uninformed. Such meritless and meretricious reviews only bring into question the integrity of the reviewers. As I am quick to point out, these people were in the minority of reviewers. Nevertheless they represent a set of people that should be excluded from any serious reviewing of just about anything, but especially any interdisciplinary research.
In a given focused discipline, like nuclear physics, most often authors get pretty fair reviews, albeit this process can get strained a bit when a reviewer's own grants and contracts are threatened. In martial arts you can at least try to defend yourself face-to-face against your opponent -- until you get to a high enough rank that you have to deal with their politics!. Interdisciplinary research requires much more of reviewers than just extracting their professional opinions on their own documented expertise in a given specialty. They have to be honest with themselves, and often with their colleagues, on just what they know and what they do not know. Good common-sense judgment (intuitive and probabilistic, tempered by analysis and experience), separate from previous expert knowledge often is required. Interdisciplinary reviewers must be prepared to acknowledge that they are not capable of reviewing a particular paper or project. My own experience with the frequent failure of interdisciplinary reviewing has led me to take care to honor this practice, and I often review papers and contracts for several institutions.
In Science, as elsewhere, opinions issued as dogma spewed by tyrants must be met by equal and opposite forces of reason and experimental data.
The only way to preserve integrity is to be true to your own intuitions and analyses, albeit this self-discipline most certainly will not make everyone happy.
26 March 2011
Every since I got my skull cracked open by a spoon during an argument over a red truck when I was about two years old, I've had problems holding on to chains of thought. I quickly learned to compensate by "thinking" in overlapping patterns, so that whenever such a lapse occurs, I just about always can quickly reconstruct my chain of thought. At a certain age, like mine at 70, these are often described as "senior moments," but I have had these moments all my life. I think this has turned into a asset, making me very creative in all my endeavors, as I uncover new patterns of information relying on such processes more than most people do, instead of having to be led by logic.
$Id: ingber_projects.html,v 10.406 2021/04/17 20:09:28 ingber Exp ingber $