In order to talk about my research topic we need to set some ground rules regard­ing the sub­jects I am going to cover in this site. Of course I could set up a com­plete course on infor­ma­tion the­ory, con­vex opti­miza­tion and ran­dom processes just to talk about chan­nel capac­ity, but it’s not the idea.

In this post, I am going to cover some basic def­i­n­i­tions of the topic of infor­ma­tion the­ory. This is the first post of a three part series where I am going to cover the basic ideas behind dig­i­tal com­mu­ni­ca­tion (here), infor­ma­tion mea­sures and their role for study­ing com­mu­ni­ca­tion chan­nels (part 2), and source mod­els (part 3). This is, by no means a com­plete descrip­tion, but rather a sum­mary of the main def­i­n­i­tions and results.
Con­tinue read­ing →