The model simulates the information source. This means that you can
*pretend* it is the source and use that to design a system. The
equiprobable source contains the information we are interested in, the
model contains garbage information which resembles in its distribution
the information we are interested in.
Anything real that is random is modeled as noise instead of an
information source (see my previous post on the channel definiton).
The difference is that a real source is modeled with a random one, but
a random source is modeled as noise.
A model hardly has the same properties of what is being modeled. You
can model an atomic explosion by computer, tell me what properties of
the explosion does the computer program have?
You can PRETEND that a model has the properties of the real thing, but
they almost NEVER actually do. A model that has the properties of the
thing being modeled is no longer a model but the real thing itself.
> >
> >>
> >> Now, random is another word which leads to great confusion. Not
> >> only is the technical meaning different from that in everyday
> >> conversation, the technical meaning can also vary depending on
> >> the technical field. To illustrate, let me construct my own
> >> quiz question motivated by part of your quiz above, namely
> >>
> >> "... a random source contains the most information since all
symbols
> >> are equiprobable" -- Brad's quiz
> >>
> >> Suppose you have a pair of fair dice. You roll these and then
> >> compute the sum of the two results. This sum is the output of our
> >> process, the possible outcomes ranging between 2 and 12.
> >> Now for the quiz:
> >>
> >> a) is this a random process?
> >>
> >> b) are all outcomes equally probable?
> >>
> >> Now let me use this quiz to illustrate a point. This is easiest
> >> if we associate a letter to each of the possible outcomes above,
> >> 2=A, 3=B, 4=C, 5=D ... 12=K. Now suppose we actually generate
> >> a "message" by the method suggested above. We keep rolling the
> >> die and recording A-K according to the sum we get. Now, if we
> >> were to assume each letter occurs with equal probability then
> >> we would have about 3.46 bits per symbol as the information
> >> content of a typical message. If we were clever enough to
> >> figure out the real probability distribution for this process,
> >> then we would be able to compress the information content to
> >> about 3.27 bits per symbol. All this has nothing to do with
> >> whether or not our message is "meaningful".
> >
> >This is very true. But if you look for a superior model you find
> that what
> >is being transmitted doesn't have to be. ie random numbers can be
created
> >locally instead of being sent. Therefore stop sending the random
> numbers and
> >generate them locally. now information being sent is zero.
> >
>
> yes, but only because nothing is being sent. But your argument
> makes no sense even from a semantic view of information. You
> say why bother sending it if it is random, I say why bother
> generating it locally if its random. IOW, if it has no value then
> there is no point genrating it. What is the point of generating
> something with no information content? Surely one doesn't need
> an algorithm to do that.
>
> >This is a good example of a MODEL of a source, ie a source that
> behaves LIKE
> >a random source. Nobody would actually transmit the random data
> if that is
> >actually what it is.
> >
>
> Not true. I have in the past generated results by the tossing
> two dice and getting the sum procedure and I have transmitted
> them to this group. I have also generated data from other
> stochastic processes and transmitted the data to this group.
> You say its more efficient to generate results locally, I say
> otherwise. Had I told people how to generate the results instead
> of sending them, no one would have actually done it and I wouldn't
> have been able to get my point across. So, transmitting the
> data was the most efficient way of communicating my message.
The fact that people would not have done it does not reflect on
whether or not it was more efficient in terms of information theory.
Sending random results does not provide any information in itself, it
may however provide information indirectly.
For example rolling a dice and sending the result is valid if the
information desired is the position of the dice. This is very
different from sending a random number because it is sending
information about a physical object.
Also information theory does not begin to deal with recalcitrant
readers who refuse to do experiments :P
>
> [...]
>
> >>
> >> OK, one last comment. Above you wrote:
> >>
> >> "Information theory does not ascribe meaning to information.
> >> It does however ascribe NO MEANING to any randomness or noise.
> >> Do you underand this?" -- Brad
> >>
> >> A fundamental result from algorithmic information theory (AIT)
> >> is that it is impossible to prove that any particular sequence
> >> is random. This is very interesting in view of the fact that
> >> the vast majority of all the possible sequences *are* random.
> >> >From this it would seem that what you suggest above is
> >> impossible.
> >
> >Not at all. It was stated that "random mutations increase
> information" Now
> >from this it is stated that the source is random, how can that be
hard to
> >work out?!?
> >
>
> But you are just illustrating my point. "random" as in random
mutation
> does not mean "random" as used in either statistics or in
information
> thoery.
Well what does it mean then?
What you are getting at is not very clear to me here.
>
> >In your above example once I know dice are being rolled it isn't
hard to
> >conclude the data is random is it?
> >
>
> It must not be so easy as you think since, as a matter of fact the
> data is not random. You neglected to answer one of the two questions
> above. The symbols A-K will not occur with equal probability in a
> typical sequence generated by that stochastic process. Therefore,
> a typical sequence is not random.
Random processes do not need to be equiprobable. The fact that rolling
a sum of 2 is less likely than rolling a sum of 7 hardly makes a dice
roll less random. As far as I am concerned "random" and "equiprobable"
are two entirely different terms.
>
> >How hard it is to tell if something is random does not mean
> anything if the
> >debate is about random changes. As I have previously said, once
> we know it
> >is random we can treat it accordingly.
> >
>
> But you've already blundered Brad. Random mutations are not random
> in the sense that you are using the term.
Please explain what you mean here. I have no idea what you are getting
at, what kind of definiton of random are you using?
--------------------------------------------
Brad Jones
3rd Year BE(IT)
Electrical & Electronic Engineering
University of Western Australia