This is a reply to Brian. But would Chris Cogan please take a look at the
last few paragraphs.
From: Brian D Harper <bharper@postbox.acs.ohio-state.edu>
>At 02:39 PM 9/6/00 +0100, Richard wrote:
>>From: Brian D Harper <bharper@postbox.acs.ohio-state.edu>
>>
>>
>> >At 01:57 PM 8/29/00 +0100, Richard wrote:
>>
[...]
>> >Generally speaking, a pattern is
>> >what one uses to compress the string. Generically, this
>> >kind of agrees with the everyday notion of pattern, but
>> >there may be exceptions. Humans seem particularly
>> >good at "seeing" patterns which aren't really there. Likewise,
>> >patterns in the sense we're speaking of here may not be
>> >obvious at all.
>> >
>> >You mention above there are three possibilities. Actually
>> >from this point of view there is a continuum of possibilities,
>> >however, purposeful is not included. Purpose is subjective,
>> >it can't be measured. The reason for the continuum is that
>> >the notion of randomness admits of degrees. Something
>> >may be very nearly incompressible but not quite. One
>> >can make this notion precise by introducing per cent
>> >compressible. Something only 5% compressible is
>> >nearly random. Something 95% compressible is very
>> >ordered.
>>
>>Yes, algorithmic randomness (as you've defined it) admits of different
>>degrees. Statistical randomness does not, strictly speaking (although
>>different degrees of variability are sometimes referred to as degrees of
>>randomness).
>
>Here is a point of contention, I think. I cannot understand referring
>to an entire process as random when that process actually contains
>many elements, only a few of which are actually random. Here I
>mean random in the statistical sense.
OK, then you've understood what I mean by random, and we're agreed that
processes can be random in the statistical sense.
>Glenn Morton (who is no longer with us) was always talking about
>Sierpinski's Gasket (SG). Here there is a stochastic element yet there
>is also a pattern which always forms. In view of the latter, I
>would be more inclined to call this deterministic. It is a matter
>of degree, of course, but the overall effect seems deterministic to me.
> From the algorithmic point of view, we would call this (SG) simple since
>it is the result of a very short algorithm.
But why do we care about whether it's simple or not? Coin flipping and
evolution are processes, and the question is whether these processes are
random, not whether they're simple or patterned.
>Anyway, using your notions it seems we would be forced to
>refer to the engineering design process that I referred to earlier
>as a random process since it contains a stochastic element.
Yes.
>But the main point for me in all of this is an attempt to
>associate meaning, purpose etc. with a technical word
>such as random, no matter how its defined. For example,
>even if I were to concede for sake of argument that evolution
>is a random process, I still would insist that this says nothing
>about purpose.
I'll come back to this once we've agreed that the relevant concept of
randomness is statistical, not algorithmic.
>> >In my view, evolution fits on this scale towards the upper
>> >end. It is not random but highly complex.
>>
>>Having just established that algorithmic randomness is a matter of degree,
>>what do you mean by "it's not random"? Relative to what?
>
>Well, as usual I'm thinking of the results of the process. If one looks
>at the fossil record, say, one will find all kinds of patterns. How
>can we infer common descent without these? So, to clarify,
>I would say that the results of evolution are clearly not (algorithmically)
>random.
Well, you still haven't answered my question. If algorithmic randomness is a
matter of degree, than it doesn't make any sense to simply say that a result
is algorithmically random. You need to say what this is relative to, or
position it on some sort of scale. Since you've associated algorithmic
randomness with compressibility, you might say that the results of evolution
are X% compressible. But then what is the significance of any particular
figure?
>> >I don't know if this helps. I think our key difference is the
>> >process versus result distinction.
>> >
>> >Oh, I almost forgot to mention the "dilemma" which Chaitin
>> >solved. This is usually described something like this. Suppose
>> >you have the following two strings which supposedly represent
>> >tosses of a fair coin. 1=heads, 2=tails:
>> >
>> >(A) 0101010101010101010101010101010101010101010101010101010101010101
>> >
>> >(B) 1110101010010010101001010011000111100111100110011111000011010011
>> >
>> >Is either of the results more surprising than the other? From the point
of
>>view
>> >of probability theory we cannot really say since the probability of any
>> >specific
>> >sequence is the same.
>> >
>> >What we learn from the above considerations is that if a number is
>> >defined to be random in view of the process by which the number is
>> >produced, then an obviously ordered result such as the first sequence
>> >above must be considered random since the random process of coin
>> >flipping produces that result with a probability equal to that of any
>> >other specific sequence.
>>
>>To clarify, you are talking about statistical randomness here, right?
>
>Yes.
>
>> >In the words of Gregory Chaitin "The conclusion
>> >is singularly unhelpful in distinguishing the random from the orderly.
>>
>>But what does Chaitin mean by random here? Since he hasn't yet introduced
>>his algorithmic randomness, he must mean statistical randomness. So his
>>objective is to detect statistical randomness.
>
>No, that is not the objective. The objective is an objective and intrinsic
>measure of order and complexity. Random lies at the upper end
>of the complexity scale. How does the theory of probability distinguish
>these if a stochastic process can produce both ordered and random
>results? Its not like we're throwing out the theory of probability here.
>What we're doing is carefully separating two issues. In this way,
>the combination of the two theories will be made much stronger.
>
>Let's go back to my coin tossing example. By combining algorithmic
>information theory and probability theory we can overcome the
>difficulty alluded to.
If "random" and "orderly" are defined in terms only of the compressibility
of the end result, then how is probability theory relevant to distinguishing
between them?
>What we can say is that a stochastic process
>involving tossing a fair coin is extremely unlikely to produce any
>patterned result. Not just the specific result (a) but *any* patterned
>result. The unlikeliness of this can be made as arbitrarily close
>to zero probability as you like by increasing the length of the sequence.
>This gives justification for our being surprised by (a).
>
>> >Clearly a more sensible definition of randomness is required, one that
>> >does not contradict the intuitive concept of a 'patternless' number."
>> >(Chaitin, G.J. (1975). "Randomness and Mathematical Proof," <Scientific
>> >American>, 232 (May 1975):47-52.)
>> >
>> >As might be expected from the above, the "more sensible definition
>> >of randomness" is Chaitin's :).
>>
>>But it is *not* sensible to redefine a word (totally changing its meaning)
>>just because you are having difficulty detecting the presence of the
>>phenomenon.
>>
>>Effectively, Chaitin is saying that we can't definitively detect
statistical
>>randomness, so he defines a measure which we *can* detect, and,
confusingly,
>>calls this randomness too. But what use is this measure, unless it's used
to
>>detect statistical randomness? Why do we care whether a set of data is
>>patterned or not? And, if algorithmic randomness *is* used to detect
>>statistical randomness, then it is really statistical randomness that
we're
>>interested in, and algorithmic randomness is just a marker for it.
>
>>Also, what relevance does algorithmic randomness have to our original
>>subject? We were talking about whether random variation and natural
>>selection (or evolution) are random processes. But you can't talk about
the
>>algorithmic randomness of a process. You can talk about whether
contemporary
>>organisms exhibit a greater or lesser degree of algorithmic randomness
(i.e.
>>they are more or less patterned), but what's the relevance of that?
>
>OK, I think things have become confused on the issue of process
>and result. Speaking as an experimentalist :) I have to say that
>the results are the most important aspect of science. Trying to detect
>patterns in data and thus compress data into theories is, perhaps,
>the most essential aspect of what scientists do. This particular
>application is, BTW, the motivation behind the development of
>algorithmic information theory for one of its co-founders, Solomonoff.
I don't doubt that describing patterns is important. But, like it or not,
scientists do go further than this and attempt to give *explanations*, i.e.
they go beyond observing correlations, to making claims about cause and
effect.
Let me quote from a book I have about the nature of science (probably not
the best book on this subject, but useful nevertheless):
"Whatever else science is used for, it is explanation that remains its
central aim. Recognition of this has led some philosophers to draw a
distinction between what the American philosopher George Gale has called
'cookbook science' and 'explanatory science'. The contrast recognizes that
science consists of two distinct steps, namely the accumulation of empirical
observations (packaged in the form of generalizations) and the invention of
explanations that tell us why these generalizations exist."
--Robin Dunbar, "The Trouble with Science", p. 17.
>As an example, consider the mountains of data that Kepler got
>from Tycho Brahe. The tremendous compression of that data into
>first Kepler's and then Newton's laws tells us something.
In formulating his theory of gravity (for example), Newton was implicitly
asserting that the process of movement of bodies under gravity is a
deterministic one. On the other hand, if I'm not mistaken, quantum theory as
serts that radioactive decay is a statistically random process.
>The compression of available data into the theory of evolution is
>not so great and not so elegant. Nor is it complete. This also
>says something. There is no proof of course. Above I gave an
>example of strong statements one could make if one combines
>algorithmic and probability theory. But before you can make such
>statements you have to know the probability distribution. To say
>that there is one begs the question, of course.
No, we don't have to know the probability distribution in order to say that
there is one. Suppose I roll 10 dice, multiply the highest 5 rolls together
and divide by the lowest 5 rolls. I have no idea what the probability
distribution of the result is, but I'm sure there is one.
If all we have is a description of a state, and no knowledge of the process
that led to it, then we may be able to draw a conclusion about whether that
process was a statistically random (probabilistic) one. The way we draw that
conclusion may be on the basis of the algorithmic randomness of the state,
but we would still be drawing a conclusion about the statistical randomness
of the process. And our conclusion may be incorrect, but that's the nature
of science. You said that you prefer the algorithmic concept of randomness
because it's definitive. But drawing provisional conclusions about the
underlying processes is part of science, and the fact that we may get it
wrong shouldn't stop us from doing it.
>Before I start rambling on too much, let me try to summarize
>what I think are our two most important disagreements.
>
>1) Calling the entire process of evolution random because it
>contains some random elements. In particular referring to
>selection as random. True, individual organisms and even
>species are guaranteed nothing due to accidents. Nevertheless,
>the overall effect of selection is one of ordering.
>
>2) Associating any technical definition of randomness with
>lack of meaning or purpose. This really is the most important
>issue for me and is what prompted my original response to
>Bertvan.
I note that you don't mention, in these points of disagreement, our
difference about whether we should be considering statistical or algorithmic
randomness. I think this is indicative of the fact that the issue of
algorithmic randomness is an unfortunate red herring. But I would like to
make sure we've got this red herring out of the way before proceeding to
discuss the two points above.
Supporters of the theory of evolution, from Darwin on, have talked about
"random mutation" (or "random variation" to be more general, but I'll stick
to "random mutation" for now). We need to settle what they mean by this.
First, this term refers to individual events or collections of events
(which I collectively refer to as processes). You've yet to explain what it
means for an event or process to be algorithmically random. As far as I can
see, algorithmic randomness is a static property.
Second, let's note that the concept of algorithmic randomness was only
defined relatively recently, while the concept of statistical randomness
goes back much further, probably at least to the time of Darwin. I haven't
noticed evolutionary biologists or geneticists redefining "random mutation"
in terms of algorithmic randomness, so I think it's only reasonable to
assume that "random mutation", in the context of the theory of evolution,
means statistical randomness.
While dropping the red herring of algorithmic randomness will help, I think
we need to go further in clarifying what is meant by "random mutation". In
theory, a mutation event could be probabilistic but still biased in favour
of greater reproductive fitness, and this would not be in accordance with
the theory of evolution. Both you and Chris Cogan have brought up the point
that the random in "random variation" means random with respect to utility.
I agreed with this. However, on further thought, I don't think this is clear
enough. I think a better formulation would be "uncorrelated with utility",
i.e. the probabilities of different mutations are uncorrelated with their
utilities. (For the moment, I won't go into what we mean by utility, but
this is an important issue that I'll address later.)
This also clarifies the issue of selection. Natural selection *is* random,
as I've said before, because it includes a probabilistic element. However,
it is also correlated with reproductive fitness. So we can say that mutation
is uncorrelated with utility but natural selection isn't. This is a lot
clearer than saying that mutation is random but natural selection isn't.
So can we agree that, according to the theory of evolution, random mutation
is uncorrelated with utility, in the above sense?
Question to Chris Cogan. If one takes a deterministic view of nature, as I
think you do, then what does it mean to say that mutations are uncorrelated
with utility (or "random with respect to utility" to use your expression)?
If probability is only a function of lack of knowledge, it has no meaning in
the absence of an observer, and evolution does not require an observer. So
we can't talk about the correlation between the probabilities of mutations
and their utility. Perhaps some sort of frequentist interpretation is
possible, but I'm having difficulty formulating one.
Richard Wein (Tich)
This archive was generated by hypermail 2b29 : Sat Sep 09 2000 - 07:34:25 EDT