Category Archives: ICM

ICM Final: Overlooked News

Final Project Page

This project is based on the talk by Ethan Zuckerman in Applications, Fall 2008. He postulated that the world was becoming more insular because news sources focused so tightly on a few small areas, while missing the news in others. Overlooked News uses the New York Times to chart out what news articles are being reported right now, and what news articles the New York Times has chosen to overlook.

Final Project Page

Advertisements

Some notes for an ICM final

So here’s my little midterm newsreader interface. Right now it doesn’t process any news. And it doesn’t find the news that’s missing and fill it in for you. I would like to change that for the final, with a database and PHP. Neither of which I know how to code.

Until now! I have my database set up, good to go, with tables for Locations, Countries, NYT Article Info, and Overlooked Article Info.

Here’s a PHP Script that reads out what’s in the NYTimes table, concatenated with the appropriate Lat and Long information from the Locations table, and changed into an XML feed for Processing to read:

http://www.binaryspark.com/reader/articleHarvester.php
(look at source if you can’t read the xml)

And here’s one that does the same for the Overlooked information. It gets it’s Lat and Long information by concatenating with the Country table.

http://www.binaryspark.com/reader/articleHarvester_overlooked.php
(look at source if you can’t read the xml)

Now to the other side of it, here’s a php script that pulls in NYTimes articles and cleans them up.

http://www.binaryspark.com/reader/articleAssigner.php

Problem number one I can see already is the New York times is really really America-centric. I thought I was making a snide comment on their insularity, but they make a much better one than I ever could. At any given time there’s no more than a dozen international countries being officially covered by news (instead of features). I may have to extend the 24 hour timeline to two or even three days just so it doesn’t become ridiculous.

Anyway, I’d also love to use this info to somehow output an “insularity index” – a way to measure what percent of the world is being covered. But then there’s the problem I mention above that so little is being covered at any given time, it’s gonna be tough to get a statistical difference day by day. So.

Achievements in php

There are times when I have a great sense of achievement from achieving nothing much. For example. Here’s a little database, populated with some fake data. And here’s a little PHP script that gets the info from one of the four tables in that database. It’s so easy a drunk 2nd grader could do it, but I never have – databases or php. go me!

http://www.binaryspark.com/reader/articleHarvester.php

And here’s the code I used for it:

<?php
//opening the connection:
$dbhost = 'localhost';
$dbuser = 'name;
$dbpass = 'pass';
 

$conn = mysql_connect($dbhost, $dbuser, $dbpass) or die                      ('Error connecting to mysql');

$dbname = 'binarysp_ICM';
mysql_select_db($dbname);

 

//actual work!
$query  = "SELECT city, country, headline, blurb, url FROM NYTArticle";  //the actual sql statement
$result = mysql_query($query);        //storing the result of $query in $result

 

echo "<html><head></head><body>" ;     //print out an html head tag

while($row = mysql_fetch_row($result))    //iterate through the rows of $result
{
    $city   = $row[0];        //get the particular item at each cell of the row
    $country = $row[1];
    $headline = $row['2'];
  $blurb= $row['3'];
  $url= $row['4'];

 

    echo "$city <br>" .          //write it all out as one big old line.  I'll actually use xml tags here.
         "$country <br>" .
         "$headline <br>" .
     "$blurb <br>" .
     "$url <br><br>" ;
 
}

 

echo "</body></html>";          //close the html tag

//closing the conection
mysql_close($conn);
?>  

ICM Midterm: A newsreader that’s pretty

This here is my midterm for ICM. The original concept was a newsreader that pulls in articles from the NYTimes and maps them on a map using Lat and Long. Then the idea is that it finds countries that aren’t receiving enough coverage a la Ethan Zuckerman and pulls in supplemental material from their newspaper feeds to give the user a better rounded picture about what’s happening in the world.

Which is to say, that’s actually my idea for the final. This is only the midterm, so allow me to present the visual interface for this monster. It’s currently hardwired with some data in the arrays it will eventually download from the database (’cause processing just can’t read xml feeds fast enough) after already calculating the information using PHP (’cause…see above. Incidentally, I dont know php).

Right now the Database Handler organizes the articles it receives into an array of map Objects. Each object knows how to display itself in various ways – on a map and as a ‘link’. Clicking on either method produces an associated article, complements of the Scenery Class, which also takes over a number of MapItem’s duties even though it’s not strictly modular. Sometimes usability takes precedence over true object orientation, or something.

For the final, besides getting the rss and url parsing to work along with the database input every minute or so, I will need to find a way to display a much longer list of cityCountry’s, along with the overlookedCountry’s list. Much html-like visuals will ensue, I’m sure. It would be nifty if I could find some more interesting information-infused animation in there too. And who knows, maybe by then NYTimes will have a chance to get it’s much rumored API up, tho I’m not holding my breath.

Link to Midterm.

Coin flippin’

So here’s a quarter, all wired up for a night out on the town. It’s got two light sensors, one on each side, feeding into the Arduino. That info then heads over to Processing where the side facing up at any given moment is displayed. I coated the quarter in clear nail polish first to cut down on sparky badness.

And here’s the code:

import processing.serial.*;     // import the Processing serial library
Serial myPort;                  // The serial port

PImage face;

PImage heads;
PImage tails;

void setup() {
  size(200,200);

  heads = loadImage("heads.jpg");
  tails = loadImage("tails.jpg");

  face=heads;

  // List all the available serial ports
  println(Serial.list());

  myPort = new Serial(this, Serial.list()[16], 9600);

  // read bytes into a buffer until you get a linefeed (ASCII 10):
  myPort.bufferUntil('\n');
}

void draw() {

    image(face, 0, 0); //image defaults to heads

}

// serialEvent  method is run automatically by the Processing applet
// whenever the buffer reaches the  byte value set in the bufferUntil()
// method in the setup():

void serialEvent(Serial myPort) {
  // read the serial buffer:
  String myString = myPort.readStringUntil('\n');
  // if you got any bytes other than the linefeed:
  if (myString != null) {

    myString = trim(myString);

    // split the string at the commas
    // and convert the sections into integers:
    int sensors[] = int(split(myString, ','));

    // print out the values you got:
    for (int sensorNum = 0; sensorNum < sensors.length; sensorNum++) {
      print("Sensor " + sensorNum + ": " + sensors[sensorNum] + "\t");
    }
    // add a linefeed after all the sensor values are printed:
    println();

    if (map(sensors[0], 10,75,0,100) > map(sensors[1], 10,65,0,100)) {
     face=heads;
    println ("one");

    }
    else{
      face=tails;
      println ("two");
    }
  }

}

Taking the temperature of the blogosphere

Now here’s a nifty little piece of code, done in Processing in conjunction with Vital. This applet queries the xml feed for every Live Journal entry being posted each moment. It chops it up into words based on spaces and punctuation, and searches the subsequent array for the words “happy” and “sad”.

As if that weren’t enough, it then plots its findings as two sine waves that update in real-time as each new post from around the world comes in. The result – you can tell how the blogosphere is feeling at any given moment.

The initial load time is a little long, so there’s a Loading message that appears the first time.

…and here’s the code!

RSSGrabber feed;
float inc = TWO_PI/90;
color happycolor = color(0, 255, 0);

float happy_x = 0.0;
float happy_y = 0.0;
color sadcolor = color(255, 0, 0);

float sad_x = 0.0;
float sad_y = 0.0;
PFont f;

String happyword = "happy";
String sadword = "sad";

String requestURL = "http://www.livejournal.com/stats/latest-rss.bml";

int counter;  //This will display the loading message only once

void setup() {

  size(800,600);
  background (0);
  f = createFont ("Arial",20,true);

  textAlign(LEFT);
  smooth();

  feed = new RSSGrabber(happyword, sadword, requestURL);
 counter=1;
}

void draw() {
  background(0);

  if (counter ==1) {
  textFont (f);
  text("Loading.  This may take up to a minute", width/3, 20);

  counter ++;
}


  println ("test");

  drawhappy ();
  drawsad();
  fill(255);

  println ("test1");
  happy_y = 0.0;
  sad_y = 0.0;

  textFont (f);
  feed.requestBlogs();
}

class RSSGrabber  {

  String Searchterm1, Searchterm2, URL;

  String [] URLarray;  //array of parsed text ready for searching

  int numberFound;  //the final count of how many instances were found

  String delimiters;

  int happy;
  int sad;

    RSSGrabber (String tempterm1,String tempterm2, String tempURL) {
    Searchterm1=tempterm1;
    Searchterm2=tempterm2;
    URL = tempURL;

    delimiters = " ,.?!;:[]";


    int numberFound = 0;

    }

void requestBlogs(){

    String[] lines = loadStrings(URL);  //brings url in as an array of strings
    String rawtext = join(lines, "" );    //joins up array into one long string
    URLarray = splitTokens(rawtext,delimiters);    //parses string up into an array, one word each cell


    happy = 0;
    sad = 0;
    instanceFinder(URLarray, Searchterm1, Searchterm2);


  }

  // A function that returns if an item is found
  void instanceFinder(String [] tempURL, String tempsearchterm1, String tempsearchterm2) {

    String theword1 = tempsearchterm1;
    String theword2 = tempsearchterm2;

  int total = 0;
  for (int i = 0; i < URLarray.length; i ++ ) {
    if (theword1.equals(URLarray[i])) {
      happy ++;
    }
    if (theword2.equals(URLarray[i]))  {
      sad++;
    }

    }  //end for loop

  println (happy);
  println (sad);

  }

}

void drawhappy() {
noStroke();
smooth();
fill (happycolor);
for(float i=0; i<width; i++) {
  ellipse (i + happy_x, height/3+2*sin(happy_y)*feed.happy,10,10);
  happy_y = happy_y + inc;
}
}

void drawsad() {
noStroke();
smooth();
fill (sadcolor);
for(float i=0; i<width; i++) {
  ellipse (i + sad_x, (height/3)*2+2*sin(sad_y)*feed.sad,10,10);
  sad_y = sad_y + inc;
}
}

Visual Clock

Yes indeed, it’s time for another Processing Applet. This one’s a Visual Clock – the letters at the right change color from black to green as they count out the milliseconds, seconds, minutes, and hours that make up a day. The display area reads out a bunch of visual feedback to give the position of the sun or moon, lightness of sky, number of stars. Feeling Impatient? Use the slider at the bottom to see what the world will look like at some other time.

http://itp.nyu.edu/~zf213/icm/Assign_4/applet/