Gourt > Science > Math > Applications > Information Theory (8)
- The topic of this article is distinct from the topics of Library and information science and, largely, from Information technology.
Information theory is a discipline in applied mathematics involving the quantification of data with the goal of enabling as much data as possible to be reliably stored on a medium and/or communicated over a channel. The measure of data, known as information entropy, is usually expressed by the average number of bits needed for storage or communication. For example, if a daily weather description has 3 bits of entropy, then, over enough days, we can describe daily weather with an average of approximately 3 bits per day (each bit being a 0 or a 1). Applications of fundamental topics of information theory include ZIP files (lossless data compression), MP3s (lossy data compression), and DSL (channel coding). The field is at the crossroads of mathematics, statistics, computer science, physics and electrical engineering, and its impact has been crucial to success of the Voyager missions to deep space, the invention of the CD, the feasibility of mobile phones, the development of the Internet, the study of human perception, the understanding of black holes, and numerous other fields.
Overview
The main concepts of information theory can be grasped by considering the most widespread means of human communication, language. Two important aspects of a good language are as follows: First, the most common words (e.g., "a," "the," "I") should be shorter than less common words (e.g., "benefit," "generation," "mediocre"), so that sentences will not be too long. Such a tradeoff in word length is analogous to data compression and is the essential aspect of source coding. Second, if part of a sentence is unheard or misheard due to noise — e.g., a passing car — the listener should still be able to glean the meaning of the underlying message. Such robustness is as essential for an electronic communication system as it is for a language; properly building such robustness into communications is done by channel coding. Source coding and channel coding are the fundamental concerns of information theory. More on [ Information theory ]
Theoretical Computer Science
Complex Systems :: Applications
A new "Information Theory" so I've heard, defines a new c=186000mi/sec according to "the rules." INTERESTING?!
StevenDHill4102 (Steven D Hill) Sat, 07 Nov 2009 18:56:12 -0000
A new "Information Theory" so I've heard, defines a new c=186000mi/sec according to "the rules." INTERESTING?!
#aasl2009 i hope we are going to get more practical information along with theory
sharamy (Sharon Amy) Sat, 07 Nov 2009 15:48:17 -0000
#aasl2009 i hope we are going to get more practical information along with theory
Information Theory :X
Funnnny (Funnnny) Sat, 07 Nov 2009 15:29:56 -0000
Information Theory :X
The closest thing to pre-web "content delivery" is information theory. But that's the engineering side... that makes the web possible. ;)
tcarmody (Tim Carmody) Sat, 07 Nov 2009 15:15:09 -0000
The closest thing to pre-web "content delivery" is information theory. But that's the engineering side... that makes the web possible. ;)
@publichistorian yeah, the closest thing you get to it is the generalized theory of information, cybernetics, etc. - I'll add that
tcarmody (Tim Carmody) Sat, 07 Nov 2009 15:12:47 -0000
@publichistorian yeah, the closest thing you get to it is the generalized theory of information, cybernetics, etc. - I'll add that
http://tinyurl.com/yc8kxml Madcow's 5x5 and Training Theory and Information Site
catch_down (catch_down) Sat, 07 Nov 2009 07:19:12 -0000
http://tinyurl.com/yc8kxml Madcow's 5x5 and Training Theory and Information Site
A Discipline Independent Definition of Information - An article about some sides of Information Theory written by Robert M. Losee. Also there is link to pdf-version of article.
Meta Description: [ A Discipline Independent Definition of Information1 ]
A Short Course in Information Theory - Eight lectures on information theory by David MacKay.
Algorithmic Information Theory - Open problems and links to software.
Basics of Information Theory - A brief introduction to information theory by David S. Touretzky of Carnegie Mellon University.
Biosemiotics - A comprehensive site dedicated to biosemiotics, an interdisciplinary science that studies communication and signification in living systems.
404
Entropy on the World Wide Web - To promote the appreciation, understanding, and applications of entropy in its many forms.
Entropy on the World Wide Web - A collection of links to on-line expository articles on entropy and theory of information, maintained by Roland Gunesch (University of Hamburg).
Meta Description: [ The purpose of these pages is to promote the appreciation,understanding, and applications of entropy in its many forms. Here you will finda collection of links to on-line expository articles and textbooks concerning the role of entropy ininformation and coding theory, dynamical systems, the theo... ]
404
Information Theory - A basic introduction and history of information theory from Bell Labs.
Meta Description: [ Our Apologies ]
Information Theory - Site promotes activities in the areas of information and communications theory in Australia. Information about conferences and other such stuff.
Meta Description: [ Information Theory ]
Information Theory and music - An article about Information Theory and music. Definition of Information Theory, Probability, Markov Chains.
Information Theory Resources - Small collection of links about Information Theory and Claude Shannon.
Introduction To Algorithmic Information Theory - An introduction to the synthesis of computation and information theory by Nick Szabo.
Linear Complexity: A Literature Survey - The linear complexity (LC) of a sequence is the size in bits of the shortest linear feedback shift register (LFSR) which can produce that sequence. The measure speaks to the difficulty of generating, and perhaps analyzing, a particular sequence.
Meta Description: [ Literature survey on linear complexity, the length
of the shortest linear feedback shift register (LFSR) to produce a
given sequence. A Ciphers By Ritter page. ]
Organs of Computation - A Talk With Steven Pinker.
Quantum Information Theory - A presentation at the undergraduate level by Michael Gibbs.
Self-Organizing Systems (SOS) FAQ - Includes a very extensive catalog of links on SOS.
Meta Description: [ Frequently Asked Questions about self-organized systems ]
The Classic Definition of Information - From the Web Dictionary of Cybernetics and Systems.
| Lecture - 8 Information Theory (Part - 1) | |
| Next Video | |
