[...]
>
>The model simulates the information source. This means that you can
>*pretend* it is the source and use that to design a system. The
>equiprobable source contains the information we are interested in, the
>model contains garbage information which resembles in its distribution
>the information we are interested in.
>
>Anything real that is random is modeled as noise instead of an
>information source (see my previous post on the channel definiton).
>The difference is that a real source is modeled with a random one, but
>a random source is modeled as noise.
>
>A model hardly has the same properties of what is being modeled. You
>can model an atomic explosion by computer, tell me what properties of
>the explosion does the computer program have?
>
>You can PRETEND that a model has the properties of the real thing, but
>they almost NEVER actually do. A model that has the properties of the
>thing being modeled is no longer a model but the real thing itself.
>
Of course the model properties are not the SAME as the real properties,
but those real properties that are of interest must be modeled
in some way. If meaning is not modeled in some way in information
theory, then information theory cannot tell you anything about
meaning.
Let me try a different approach. If I remember correctly, you
have previously agreed that information theory cannot tell
you what the meaning of a message is. Why is this? Let's recall
what your Professor said about the quiz question. He said
something to the effect that info theory is all about determining
the quantity of information in a message. So, herein lies the
difficulty. Meaning is established by mutual agreement between
two or more individuals. This meaning cannot be quantified,
i.e. expressed mathematically [**]. But information theory is a
mathematical theory. Therefore, information theory cannot
quantify (measure) meaning. OK, up to now I've just tried to
make what we seem to agree on more precise, i.e. we agree that
info theory cannot tell us the meaning of a message. The above
is my explanation as to why. So, if we agree on this point, the
final step is very tiny. If info theory cannot measure meaning
then it cannot tell you the quantity of meaning in a message and
if it cannot measure meaning it cannot even tell you whether
a message has *any* meaning at all.
[**]: Here I'm assuming that people have free will. For deterministic
ultra-reductionists it may not seem out of the question that meaning
might be quantified. But I'm sure you will not object to the
assumption that humans have free will.
[...]
>>BH:==
>> Not true. I have in the past generated results by the tossing
>> two dice and getting the sum procedure and I have transmitted
>> them to this group. I have also generated data from other
>> stochastic processes and transmitted the data to this group.
>> You say its more efficient to generate results locally, I say
>> otherwise. Had I told people how to generate the results instead
>> of sending them, no one would have actually done it and I wouldn't
>> have been able to get my point across. So, transmitting the
>> data was the most efficient way of communicating my message.
>Brad:===
>The fact that people would not have done it does not reflect on
>whether or not it was more efficient in terms of information theory.
>Sending random results does not provide any information in itself, it
>may however provide information indirectly.
>
>For example rolling a dice and sending the result is valid if the
>information desired is the position of the dice. This is very
>different from sending a random number because it is sending
>information about a physical object.
>
This just illustrates the difficulty that you are getting yourself
into. According to your point of view, exactly the same message
may or may not contain information. This is hardly a good way to
define information.
>Also information theory does not begin to deal with recalcitrant
>readers who refuse to do experiments :P
>
>>
>> [...]
>>
>> >>
>> >> OK, one last comment. Above you wrote:
>> >>
>> >> "Information theory does not ascribe meaning to information.
>> >> It does however ascribe NO MEANING to any randomness or noise.
>> >> Do you underand this?" -- Brad
>> >>
>> >> A fundamental result from algorithmic information theory (AIT)
>> >> is that it is impossible to prove that any particular sequence
>> >> is random. This is very interesting in view of the fact that
>> >> the vast majority of all the possible sequences *are* random.
>> >> >From this it would seem that what you suggest above is
>> >> impossible.
>> >
>> >Not at all. It was stated that "random mutations increase
>> information" Now
>> >from this it is stated that the source is random, how can that be
>hard to
>> >work out?!?
>> >
>>
BH:==
>> But you are just illustrating my point. "random" as in random
>>mutation does not mean "random" as used in either statistics
>>or in information thoery.
>
Brad:==
>Well what does it mean then?
>
>What you are getting at is not very clear to me here.
>
OK, "random" as in random mutation means that mutations do not
appear for the benefit of the organism. Or another way of saying
it is that mutations are not directed in such a way as to
benefit the organism. They are "random" wrt to their usefulness
to the survival of an organism.
>
>>
>> >In your above example once I know dice are being rolled it isn't
>hard to
>> >conclude the data is random is it?
>> >
>>
>> It must not be so easy as you think since, as a matter of fact the
>> data is not random. You neglected to answer one of the two questions
>> above. The symbols A-K will not occur with equal probability in a
>> typical sequence generated by that stochastic process. Therefore,
>> a typical sequence is not random.
>
>Random processes do not need to be equiprobable. The fact that rolling
>a sum of 2 is less likely than rolling a sum of 7 hardly makes a dice
>roll less random. As far as I am concerned "random" and "equiprobable"
>are two entirely different terms.
>
Good, I agree, so I think I'll finally be able to illustrate the
point I had in mind with this example which has really been my
main theme, that random is not always random. From the point
of view of probability theory a random process does not have to
have equiprobable outcomes. I would add further that equiprobable
outcomes is a special case. In general one expects otherwise.
So, what you say is perfectly correct. But what I said is also
correct when random takes its meaning from algorithmic information
theory (AIT). In this case random has nothing to do with the
process, it has only to do with the structure of the actual
sequence. If a sequence is incompressible it is random, if it
can be compressed it is not random.
So we have the seemingly bizarre situation wherein the typical
outcome of a random process gives a nonrandom result. To avoid
this confusion, many authors have implemented a different
terminology so that when they have a random process a la
probability theory they refer to it as a stochastic process.
So we can say then that the typical result of a stochastic
process with unequal probabilities is a sequence which is
compressible (nonrandom).
Hopefully this example illustrates the importance of understanding
the meaning of words as they are used in different fields and the
danger of carelessly carrying a word from one field to another
without such an understanding.
So, just as AIT random doesn't mean the same thing as probability
theory random, so also evolutionary biology random doesn't
mean the same thing as probability theory random. Whatever
conclusions one might draw based on this mistaken identity
are erroneous.
Brian Harper
Associate Professor
Applied Mechanics
The Ohio State University
"It appears to me that this author is asking
much less than what you are refusing to answer"
-- Galileo (as Simplicio in _The Dialogue_)