Talk:Shannon–Hartley theorem

message: The external link called "The relationship between information, bandwidth and noise" is dead, the data may have been moved, or even removed.  —Preceding unsigned comment added by 134.117.251.2 (talk) 02:40, 25 February 2011 (UTC)
thread_title:  Dead external link 

message: Original: The last paragraph (especially "V.90 claims a rate of 56 kbit/s, apparently in excess of the Shannon capacity") is wrong. The V.90 data rate does not account for compression. 
Edit by Mark Rejhon: This modem information has now been clarified from Mark Rejhon's edit about a month ago.  This needs some peer review.  Also, 56 kbit/s is achieved without compression, or rather, 53 kbit/s due to FCC regulations on signal level.  But this is done, without compression!  It's rather, done, via avoiding a digital-to-analog conversion step, at the telco end.   This information is widely available on the Internet, but needs to be confirmed with accurate sources.
:From my study of the Shannon channel capacity theorem, compression doesn't matter. Information is information, compressed or not. To the best of my knowledge, Shannon made no assumptions about the coding of information. This is his real genius. In general, his analysis is based on "nats", not bits (base e as opposed to base 2). Binary is a convenient coding, so most people talk about bits. As far as I know, Telcos typically sample at 8kHz with 8 bit samples. This means a Nyquist limit of 4 kHz an ideal quantization noise of about 48 dB (maybe even 54 dB if you use half of 256). If you lived 10 feet away from your central office and there was no other interference, you might get that kind of S/N. Reality is that you are probably thousands of feet from the CO and there are hundreds of other lines next to yours adding noise. I am told that 30 dB S/N is possible with BW of 3.5 kHz. This works out to about 35 kb/s. If you assume 48 dB, the channel capacity is 55.8 kb/s. Close to that mystical figure of 56 kb/s, I would think. "Principles of Communication Systems" by Taub and Shilling is a great book. Madhu
::If you have like, encyclopedia.txt, a 100 MB file, and you compress it down to 1 MB, send it in one second, and them decompress it later, you could say that you had sent 100 MB/sec.  I think that's all they meant.  They are talking about the actual transmission speed (in this case 1 MB/sec).  Apparently modems can also do on-the-fly compression, which would give effective rates greater than the actual channel capacity for easily compressed data. - Omegatron 16:38, Feb 17, 2005 (UTC)
::In my understanding, anyway, the theorem only applies to the actual data being sent, regardless of whether it is compressed data, binary, or all zeroes.  In other words, compression of the data does not count towards getting closer to the theoretical max. - Omegatron 17:02, Feb 17, 2005 (UTC)
::: I think we're saying the same thing. The important detail is the terminology. Shannon talks about information rate as opposed to bit rate (he doesn't really talk about bits). In your above example, the information rate is 1 MB/sec. If I have a 100 MB file with truly random data that cannot be compressed (try compressing a compressed file sometime ;-), then the bit rate drops to 1 MB/sec instead of 100 MB/sec. Another way to say this is that a highly compressible file contains less information than an incompressible file. Madhu
::: I agree with Msiddalingaiah; said another way, for a given coding method, data containing less information (making it more redundant) will compress more, requiring less B for a given C. Coding (non-lossy data compression if you will) effectively increases the S/N. BTW, for the example pulled out of the air by Omegatron, if there were such a 100:1 code, the effective increase in S/N would be huge, from say 0 dB at 1 Mb/s to 1204 dB (one thousand two hundred and four dB) at 100 Mb/s! Shannon's channel capacity theorem is agnostic to how the data is coded, and merely places an upper limit on the channel's capacity to carry data, for a given coding method. It does not say that there is a lossless 100:1 code. Somehow I doubt that such a code exists anywhere today, but that's not to say that it might not in the future. The CCT is *not* a  proof of existence or non-existence of coding methods. One thing is certain: if the encoding is in anyway "tuned" to the data at hand, the decoder must also contain that tuning information, which itself must have been transmitted in the channel beforehand. 
::: Brief study of Shannon's CCT reveals that as the S/N increases without bound, so does the channel capacity. Also, for zero S/N, C = B. Even though Shannon does not discuss binary data, the log2 in the formula suggests that binary representation is very fundamental, and it relates C to binary data word length (we need twice the word length to double C, which is the reality seen by anyone who has messed around with digital filter algorithms and numerical methods.) C is undefined if the noise is zero, but that is not a problem in the real world. These behaviors seem reasonable. I hope these few points may help others to better understand this topic.  —Preceding unsigned comment added by 99.40.33.249 (talk) 00:11, 16 February 2011 (UTC)
thread_title:  Modem comparison needs peer review 

message: In the "R < C" section, is C (the capacity) of the unit "bits per second", as in "my modem has 56 kbit/s"? --Abdull 06:51, 16 October 2005 (UTC)
replies: text: In strict terminology, C should be measured in "bits per channel use", which is a ratio between "bits per second" and "Hertzian bandwidth available for transmission". In this case, C is not normalized and it is in bit/s, but it is not referred to the modem speed (it is rather R): think of it as the capacity, in liters/s, of the lavabo tube (the channel) which empties the pasta pot boiling water (your modem). Cantalamessa 12:30, 8 December 2005 (UTC)
thread_title:  Comparison 

message: Isn´t there an error in the reference list: Shannon´s seminal book from the 40s is "The Mathemetical Theory of Communication", whereas "The Mathematical Theory of Information" (which may prove to be seminal as well) was written by Jan Kåhre in 2002.
timonju
:Fixed. Well spotted! -- Jheald 11:42, 12 February 2006 (UTC)
thread_title:  Error in reference list? 

message: Before the change I just made, the statement of the theorem implied that when rate equaled capacity, error could never be made arbitrarily small.  This is clearly wrong; a lossless channel can achieve its capacity rate and, in spite of its being somewhat degenerate, does fall within this framework.  The 1 December 2005 "fix" was wrong (though what it attempted to correct was also wrong).  I've fixed this so that it's clear that the R=C case is not addressed in the noisy-channel coding theorem, but someone might want to double-check my wording on this (which is also in Noisy channel coding theorem). Calbaer 23:24, 25 April 2006 (UTC)
replies: text: There's no such thing as a "lossless" additive gaussian noise channel, so whatever you did is suspect. Dicklyon 06:00, 1 September 2006 (UTC)
thread_title:  Error in noisy-channel coding theorem 

message: Since the article is named after Hartley and had almost nothing about what he showed or how it related to Shannon's capacity, I did some major edits.  I took out the irrelevant thing about the data modem, and editted the other bits.  And I tossed the alternate forms where instead of 1+S/N it used (S+N)/N, as just too obvious and redundant.  I kept the one that commonly appears in books. 
If any of this is not cool, please say so or fix it back, rather than reverting the whole lot. Dicklyon 06:00, 1 September 2006 (UTC)
thread_title: New edits

message: First Harmonic, your edits look good.  I'm assuming that since you added the Wozencraft and Jacobs citation, the equation you used is probably from there.  I'll check mine tomorrow at work to verify.  But if so, then a reference link from that sentence to the book would be in order, yes? Dicklyon 01:58, 13 October 2006 (UTC)
replies: text: Yes it is in Wozencraft & Jacobs, as I indicated in the Edit Summary (see the History page).   First Harmonic 02:55, 13 October 2006 (UTC)
replies: text: Thanks.  I should have checked your diffs and history. Dicklyon 03:30, 13 October 2006 (UTC)

text: Yes, a reference link would be fine, although I don't think it is necessary since I added W&J to the list of references at the end of the article.  Also, I have no idea how to create such a link, nor is it clear that WP has a consistent or well-established policy on how to deal with footnotes and references.  There are several different approaches in many different articles, and I am not really interested in sorting it out.  But feel free to take a stab.  First Harmonic 02:55, 13 October 2006 (UTC)
replies: text: I know what you mean.  I do feel free, but I may not take it on either. Dicklyon 03:30, 13 October 2006 (UTC)

text: Personally, I like the way that the IEEE does references in its articles, where the references are listed at the end of the article either in alphabetical order by author or in the order in which the article cites the references.  Each reference is numbered from 1 to N, and the citations within the text simply mention the number of the reference, usually enclosed within square brackets, as in [3].  But that's just me.  First Harmonic 13:57, 13 October 2006 (UTC)

text: BTW, the official WP policy is here if anyone is interested.  First Harmonic 14:00, 13 October 2006 (UTC)
thread_title: Hartley's law again

message: This page redirects from "Shannon Limit" but never once uses the phrase "Shannon Limit". That makes it not very useful for someone trying to find what "Shannon Limit" means.
Google says "Shannon Limit" appears 115,000 times but "Hartley theorem" is 534 times, including many of these being reprints of Wikipedia. (For google scholar -- i.e. recently real research -- it's 5700 to 70. The IEEE journals list 15 articles entitled "Shannon limit," one entitled "Shannon-Hartley" and none entitled "Hartley theorem"
Absent evidence to the contrary, the standard terminology appears overwhelmingly to be "Shannon limit". (Perhaps because whatever Hartley did, it was Shannon's 1948 papers that launched the research that has brought communications close to that limit).
So we need a separate article about Shannon Limit -- that's what people will be looking up and wanting to understand. If someone wants to say "Also called the Shannon-Hartley Theorem," or "builds on the work of Ralph Hartley," that's fine -- but the field calls it the Shannon Limit
:The term "Shannon limit" appears so frequently because it means so many different things.  In general, if someone says their system performs "close to the Shannon limit", they mean close to the bounds imposed by Shannon's information theory.  It is used for the capacity of a bandlimited gaussian noise channel, like the Shannon–Hartley theorem, but also for binary symmetric channels and other channels, and for source coding (lossless data compression).  One very common use is for the required energy per bit or SNR to achieve reliable transmission in the presence of noise; sort of the dual problem of finding a rate limit given an SNR.  So, I don't see how you can have an article on the "Shannon limit" since it's not a defined concept.  You have articles on information theory and this one a particular theorem of information theory.  If there's some other article that appears to be missing, by all means start it. Dicklyon 04:43, 29 October 2006 (UTC)
thread_title:  Shannon Limit 

message: A V.92 downlink can handle 56kbps in the downlink but 48kbps in the uplink. Can this difference be explained by different information capacity according to Shannon_hartley in the uplink and in the downlink? In the downlink, the modulator has digital interface, and utilizes the PCM system for sending one symbol per sample. Perhaps the inter-symbol-interference may be lower in this case, since the modem symbols are synchronized with the PCM sample instants? Perhaps the maximum signal strength is lower, since it may be hard to identify the maximum possible amplitude?  
What is the S/N in a PCM system, if there is no noice? Is there a difference between Europe and America because of different PCM systems? 
I think it is more appropriate to use the Nyquist capacity limit instead of the Shannon limit when discussing the PCM case, since we now the number of possible levels. However, note that Shannon gives the net bit rate with an ideal error correction code, while Nyquist the gross bit rate.
We may assume that that we could use all N=256 levels, resulting in a gross bit rate capacity of 2*B*log2256 = 16*B bit/s, where B is the analog bandwidth in Hertz. If we assume that the bandwidth is B=4,000 Hz (i.e. ideal filtering), the modulation would result in  according to Nyquist we could get 16*4,000 = 64,000 bit/s. If we instead assume only 3,400-300 = 3,100 Hertz Bandwidth, we would get 49,600 bit/s. In practice a frequencies outside the passband may also be utilized. The V.92 maximum downlink speed corresponds to 56,000/16 = 3500 Hertz Bandwidth.
N = 256 levels, without any noise, gives the same bit rate as the Shannon limit would give, if the SNR was 20 log10 256 = 48 decibel. 
Mange01 23:29, 30 November 2006 (UTC)
thread_title:  Can Shannon-Hartley explain different uplink and downlink speed in V.92 modems? 

message: The title and the text contain an em dash.
When a link containing that (instead of the Unicode-to-ASCII conversion) is cut and pasted to reference the article, that link appears as broken.
This is especially bad because the em dash is not the proper punctuation mark to use to seperate the guys' names anyway.  —Preceding unsigned comment added by 198.147.225.86 (talk) 18:53, 4 March 2009 (UTC)
thread_title:  Non-clickable link 

message: I removed the following piece of text:
:OnLive challenge of Shannon - Hartley "law"
:In a guest lecturer presentation delivered in December 2009 at Columbia University in New York City, OnLine(tm) executive Steven Perlman described his efforts to break the current limitation. In his words: "People talk about Shannon's Law, give me a break, it's not a law - it is a limit". He described how his company's current compression technology delivers 100X Shannon in one wireless channel. He argued that Shannon was not wrong, but that "he was asking the wrong questions".
:A video clip of the presentation was featured on CrunchGear and is located at  OnLive Presentation (the relevant section is discussed around time frame 39:05)
The reason is that it doesn't actually seem to challenge anything about the theorem. Furthermore, the presentation doesn't go into details about how this "100X Shannon" was supposedly achieved. More likely than not, the fact that compression was used (which has to be lossy or impose constraints on the input signal's entropy) wasn't taken into account when comparing the channel capacity to its theoretical limit. It's hard to say more than that without more details, but this is certainly enough reason to remove the aforementioned highly anecdotal section. 217.149.210.16 (talk) 17:45, 17 January 2010 (UTC)
thread_title:  OnLive section 

message: "Hartley's rate result can be viewed as the capacity of an errorless M-ary channel of 2B  symbols per second. Some authors refer to it as a capacity. But such an errorless channel is an idealization, and the result is necessarily less than the Shannon capacity of the noisy channel of bandwidth B". 
This doesn't sound right. Since we're talking about capacity, shouldn't Hartley capacity calculation actually result be *more* (as in more optimistic, resulting in more capacity) than Shannon? I can't give it more thought right now and don't dare to edit the article directly, sorry... just wanted to point it out.  —Preceding unsigned comment added by 80.35.240.178 (talk) 05:04, 11 October 2010 (UTC)
replies: text: The Hartley result is not a capacity, and doesn't correspond to an error-free rate.  It can be low error rate only if M is low enough, in which case the rate will be well below the capacity.  If M is chosen higher, the rate can be above capacity, but the error rate will be high.  Dicklyon (talk) 03:20, 4 September 2011 (UTC)
thread_title:  Hartley vs Shannon 

message: "(Note: An infinite-bandwidth analog channel can't transmit unlimited amounts of error-free data, without infinite signal power)"
I'm not sure that that's true.  Given an infinite-bandwidth analog channel, surely it is possible to transmit a countably infinite amount of information as an infinite sequence of harmonics, each signal scaled so the series of powers converges to a finite limit? 
Another way to see this is that noise on an infinite-bandwidth, finite power channel has to contain an infinite amount of information because if you keep the power of a noise signal fixed and increase its bandwidth without limit, the information also increases without limit.
So I'm not sure the quoted statement is true, and if it is true, some sort of citation or argument might be needed.  
(Also the comma should not be there - sorry for the nitpick.)
TomRitchford (talk)  —Preceding undated comment added 10:40, 25 November 2018 (UTC)
thread_title:  Infinite bandwidth analog channels and infinite power?