Spicing up a IBM Bluemix cloud app with MongoDB and NodeExpress

In this post I highlight the rudiments for a creating a cloud application on IBM’s PaaS offering Bluemix, using MongoDB and NodeExpress.   Clearly Bluemix allows one to fire up a cloud application with a NoSQL database in a matter of  a few hours which makes it really attractive. The NodeExpress  application was initially created using Enide Studio for Node.js  with a local Mongodb server running on my desktop. (Please see my post Elements of CRUD with Node Express and MongoDB) Once you have ironed out the issues in this local application you are ready to deploy on IBM Bluemix.

The code for this Bluemix application can be forked from bluemix-mongo from IBM Devops.

You can also clone the code from GitHub at bluemix-mongo

Here are the key changes that need to be made for running the NodeExpress Webserver with MongoDB as the backend DB

1) Webserver : Setup the port and host for the Webserver.

  1. app.js

var port = (process.env.VCAP_APP_PORT || 1337);
var host = (process.env.VCAP_APP_HOST || '0.0.0.0');
var app = express();
app.configure(function(){
app.set('port', port);

As seen above the host & port for the Webserver are obtained from the process.env variable.
2) Routes and Middleware
Setup the routes and invoke them appropriately in app.js
var express = require('express')
, routes = require('./routes')
, user = require('./routes/user')
, userlist = require('./routes/userlist')
, newuser = require('./routes/newuser')
, adduser = require('./routes/adduser')
, changeuser = require('./routes/changeuser')
, updateuser = require('./routes/updateuser')
, remuser = require('./routes/remuser')
, deleteuser = require('./routes/deleteuser')

app.get('/users', user.list);
app.get('/helloworld', routes.index);
app.get('/userlist', userlist.list);
app.get('/newuser', newuser.list);
app.post('/adduser',adduser.list);
app.get('/changeuser', changeuser.list);
app.post('/updateuser', updateuser.list);
app.get('/remuser', remuser.list);
app.post('/deleteuser',deleteuser.list);

3) Initialize MongoDB database: Create a set of 3 records when the Webserver starts as follows

if (process.env.VCAP_SERVICES) {
var env = JSON.parse(process.env.VCAP_SERVICES);
if (env['mongodb-2.2']) {
var mongo = env['mongodb-2.2'][0]['credentials'];
}
} else {
var mongo = {
"username" : "user1",
"password" : "secret",
"url" : "mongodb://user1:secret@localhost:27017/test"
}
}
var MongoClient = mongodb.MongoClient;
var db= MongoClient.connect(mongo.url, function(err, db) {
if(err) {
log("failed to connect to the database");
} else {
log("connected to database");
}
var collection = db.collection('phonebook');
//Clear DB and insert 3 records
remove(mycallback);
var user1 = { "FirstName" : "Tinniam", "LastName" : "Ganesh","Mobile": "916732177728" };
var user2 = { "FirstName" : "Darth", "LastName" : "Vader","Mobile": "6666699999" };
var user3 = { "FirstName" : "Bill", "LastName" : "Shakespeare","Mobile": "8342189991" };

  1. insert(user1,function(err,result){});
  2. insert(user2,function(err,result){});
  3. insert(user3,function(err,result){});
  4. find().toArray(function(err, items) {

});
});

3) Home Page: Setup up a Home page with the CRUD operations when the Bluemix cloud application’s route  for e.g. http://bluemix-mongo.mybluemix.net is clicked. This is shown below.

1

 

2

4) Display Users: To display the list of users the route /userlist is invoked. This function gets all the records from the collection and stores them into a toArray element, which is then used for rendering the list of uses with a ‘userlist.jade’ template

userlist.js
var MongoClient = mongodb.MongoClient;
var db= MongoClient.connect(mongo.url, function(err, db) {
if(err) {

  1. log(“Failed to connect to the database”);

} else {

  1. log(“Connected to database”);

}
var collection = db.collection(‘phonebook’);
//Get all records and display them

  1. find().toArray(function(err, items) {
  2.    log(items);
  3. render(‘userlist’, {

“userlist” : items
});
});
});

  1. jade

This template displays the list of users as a table. The code is shown below

extends layout
block content
h1= "Display the list of Users"
p
strong Firstname Lastname   Mobile
table
each user, i in userlist
tr
td #{user.FirstName}
td #{user.LastName}
td #{user.Mobile}
p
p
a(href='/') Home

Note: A link back to the Home page is included in here at the bottom.

7

 

5) Adding a User
There are 2 parts to this
a) Invoking the /newuser route to display the input form through the newuser.jade
b) Invoking the /adduser route to insert the values entered in the form. The changes are shown below
a) app.js
..
newuser = require('./routes/newuser')
adduser = require('./routes/adduser')
..
app.get('/newuser', newuser.list);
app.post('/adduser',adduser.list);

b) newuser.js
exports.list = function(req, res){

  1. render(‘newuser’, { title: ‘Add User’});

};

The newuser jade displays the input form
c) newuser.jade
extends layout
block content
h1= "Add a User"
form#formAddUser(name="adduser",method="post",action="/adduser")
input#inputUserFirstName(type="text", placeholder="firstname", name="firstname")
input#inputUserLastName(type="text", placeholder="lastname", name="lastname")
input#inputUserLastName(type="text", placeholder="mobile", name="mobile")
button#btnSubmit(type="submit") submit
p
p
a(href='/') Home

3

d) adduser.js

The adduser.js gets the mongo url from the process.env.VCAP_SERVICES and  setups up the connection to the DB and inserts the values received in the ‘newuser.jade’ form into the database

exports.list = function(req, res) {
if (process.env.VCAP_SERVICES) {
var env = JSON.parse(process.env.VCAP_SERVICES);
if (env['mongodb-2.2']) {
var mongo = env['mongodb-2.2'][0]['credentials'];
}
} else {
var mongo = {
"username" : "user1",
"password" : "secret",
"url" : "mongodb://user1:secret@localhost:27017/test"
}
}
// Set up the DB connection
var MongoClient = mongodb.MongoClient;
var db= MongoClient.connect(mongo.url, function(err, db) {
if(err) {

  1. log(“Failed to connect to the database”);

} else {

  1. log(“Connected to database”);

}
// Get our form values. These rely on the “name” attributes
var FirstName = req.body.firstname;
var LastName = req.body.lastname;
var Mobile = req.body.mobile;
// Set our collection
var collection = db.collection(‘phonebook’);
// Insert the record into the DB

  1. insert({

“FirstName” : FirstName,
“LastName” : LastName,
“Mobile” : Mobile
}, function (err, doc) {
if (err) {
// If it failed, return error

  1. send(“There was a problem adding the information to the database.”);

}
else {
// Redirect to userlist – Display users

  1. location(“userlist”);

// And forward to success page

  1. redirect(“userlist”);

}
});
});

If the insert is successful the userlist page is displayed with the new user

4

6) Updating a User & Deleting a User: Updating and Deleting users follow the same format as Adding a user.

7) index.jade The Home page is built using index.jade with a hyperlink invoking the route for each database operation
extends layout
block content
h1= title
p Welcome to #{title}
ul
li
a(href='/userlist') Display list of users
li
a(href='/newuser') Add a user
li
a(href='/changeuser') Update a user
li
a(href='/remuser') Delete a user

Tip: “Return of the Jadei : Getting the jade template right is truly an art as Jade is extremely finicky about spaces, tabs, indents and outdents(???). Creating the Jade template had me run into circles. I found out that you can debug the jade template individually by executing

C:> npm install jade -g"
and then  running
C:> jade <template name>

from the command prompt. If the result of the command is “rendered <template name>.html” then you are in luck and you can incorporate this jade template into your views folder for e.g.

C: >jade index.jade
rendered index.html

8) Push changes to Bluemix: Once the changes have been made push the changes on to Bluemix with ‘cf’ as follows

cf login -a https://api.ng.bluemix.net
cf push bluemix-mongo -p . -m 512M
cf create-service mongodb 100 mongodb01
cf bind-service bluemix-mongo mongodb01

 

The last 2 commands can also be performed through the Bluemix dashboard in which you add the mongodb service to your Node.js app/

8) Files and Logs: In the Bluemix dashboard you can check your logs in the Files and Logs

5

 

6

Important tip: Finally if the application fails to start when you  push the application with ‘cf’ for e.g.

cf push <app name> -p . -m 512M
....
.....
----> Writing a custom .npmrc to circumvent npm bugs
----> Installing dependencies
----> Caching node_modules directory for future builds
----> Cleaning up node-gyp and npm artifacts
----> No Procfile found; Adding npm start to new Procfile
----> Building runtime environment
----> Checking and configuring service extensions
----> Uploading droplet (7.6M)
of 1 instances running, 1 down
of 1 instances running, 1 down
of 1 instances running, 1 down
of 1 instances running, 1 down

or if  it crashes when you click a link then your debugging friend is

cf logs <app name > — recent
This will dump the error that was encountered either while the application was being started of why the application crashed.

You can fork this Bluemix application from bluemix-mongo at  IBM Devops or from GitHub at bluemix-mongo

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

You may also like
1. Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
2. A Bluemix recipe with MongoDB and Node.js
3. A Cloud Medley with IBM’s Bluemix, Cloudant and Node.js
4. Rock N’ Roll with Bluemix, Cloudant & NodeExpress


Find me on Google+

A Bluemix recipe with MongoDB and Node.js

Here is a tasty IBM Bluemix recipe with a dash of MongoDB and a pinch of Node.js. This posts shows the steps needed to perform basic CRUD (Create, Remove, Update & Delete) operations on the MongoDB database using REST APIs of PUT,GET, UPDATE & DELETE.

You can fork the code for the below app from Devops at mymongodb

The code can also be cloned from GitHub at mymongodb

For this,  the first  thing we need to do is to create a Webserver using Node.js as shown below

Webserver

require('http').createServer(function(req, res) {
if ( typeof mongodb !== 'undefined' && mongodb ) {
// Perform CRUD operations through REST APIs
if(req.method == 'POST') {
insert_records(req,res);
}
else if(req.method == 'GET') {
list_records(req,res);
}
else if(req.method == 'PUT') {
update_records(req,res);
}
else if(req.method == 'DELETE') {
delete_record(req,res);
}
} else {
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write("No MongoDB service instance is bound.\n");
res.end();
}
}).listen(port, host);

The Webserver users the port and host values obtained as shown above to wait for  HTTP requests.

The REST API calls are handled by individual  Node.js functions to perform the operations of insert, update, delete and select.

Insertions

The code for insertions is shown below. For this a set of 5 documents are created and then inserted using Node.js

var insert_records = function(req, res) {
var MongoClient = require('mongodb').MongoClient;
// Connect to the db
MongoClient.connect (mongo.url, function(err, db) {
//Create a collection test
var collection = db.collection('books', function(err, collection) {
//Create a set of documents to insert
var book1 = { book: "The Firm", author: "John Grisham", qty: 3 };
var book2 = { book: "Foundation", author: "Isaac Asimov", qty: 5 };
collection.remove(mycallback);
//Insert the books
console.log("Insert the books");
collection.insert(book1,function(err,result){});
collection.insert(book2, {w:1}, function(err, result) {
});
collection.insert(book5, {w:1}, function(err, result) {});
console.log('Inserted 5 books');
}); //var collection
}); // End MongoClient.connect
}; // End insert_records

Updating documents in Mongodb

For update 2 documents are changed as shown below

var update_records = function(req, res) {
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect (mongo.url, function(err, db) {
// Update
var collection = db.collection('books', function(err, collection) {
collection.update({book:"Fountainhead"},{$set:{qty:2}}, {w:1},function(err,result) {});
collection.update({book:"Animal Farm"},{$set:{author:"George Orwell"}}, {w:1},function(err,result) {});
console.log("Updated 2 books");

}); // var collection
}); //End MongoClient.connect

}; //End update-records

Deletions

The delete functions requires a callback method which is included

var delete_record = function(req, res) {
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect (mongo.url, function(err, db) {

//Deleting documents
var collection = db.collection(‘books’, function(err, collection) {

collection.remove({book:”Foundation”},mycallback);
collection.remove({book:”The Da Vinci Code”},{w:1},mycallback);
console.log(‘Deleted 2 books’);

});
}); //End MongoClient.connect
}; //End delete-records

Retrieving documents

To retrieve documents the collection.find.stream() method is used as below

var list_records = function(req, res) {
var MongoClient = require('mongodb').MongoClient;
MongoClient.connect (mongo.url, function(err, db) {
//Retrieve documents
var collection = db.collection('books', function(err, collection) {
var stream = collection.find().stream();
console.log("Printing values...");
res.writeHead(200, {'Content-Type': 'text/plain'});
stream.on('error', function (err) {
console.error(err.stack)
});

stream.on(“data”, function(item) {
console.log(item);
res.write(JSON.stringify(item) + “\n”);
});

stream.on(“end”, function() {
console.log(“End”);
res.end();
});
}); //var collection
}); //End MongoClient.connect

The connection between the Node.js & Webserver and the MongoDB is setup using the VCAP_SERVICES as follows

if (process.env.VCAP_SERVICES) {
var env = JSON.parse(process.env.VCAP_SERVICES);
if (env['mongodb-2.2']) {
var mongo = env['mongodb-2.2'][0]['credentials'];
}
} else {
var mongo = {
"username" : "user1",
"password" : "secret",
"url" : "mongodb://user1:secret@localhost:27017/test"
}
}

To get started you can fork the code for the above Bluemix- MongoDB app from Devops from mymongodb

The code can also be cloned from GitHub at mymongodb

After you have forked the code you can clone the code into a local directory on your machine.

Now use the ‘cf’ command to push the code onto IBM Bluemix as shown. In my case I named the app as mymongodb01

cf login -a https://api.ng.bluemix.net
cf push mymongodb01 -p . -m 512M
cf create-service mongodb 100 mongodb
cf bind-service mymongodb01 mongodb

Instead of the last 2 steps you can also use the Add-service in Bluemix dashboard to add the MongoDB service. (Note: You will have to check Experimental at the top and you will see the service under Data Management.) After the MongoDB service is added check if your app is running in the Bluemix dashboard

If the app is running you can check the CRUD operations on MongoDB using the SureUtils-REST API client extension to Chrome.

The CRUD operations performed are shown below

1.POST + GET

Here 5 documents are inserted in the MongoDB and then displayed subsequently

1

2

2.UPDATE + GET

Here 2 records are updated – The quantity of the book ‘Fountainhead’ is set to 2 and the author of ‘Animal Farm’ is set to George Orwell

3

4

3.DELETE + GET

In this set 2 book are deleted and the result is displayed

5

6

If all things went well you should be able to see the app running.

7

You can also get the output o the console.login Files and Logs in the Bluemix dashboard.

8

Important tip:  While executing the Buemix app if you run into problems and you app crashes with the “Health decreased” for your app and its colour turning red you can use the recent history of ‘cf” command to debug your problem

PS C:\Users\IBM_ADMIN\git\mymongodb> cf logs mymongodb01 –recent

Connected, dumping recent logs for app mymongodb01 in org tvganesh.85@gmail.com / space dev as tvganesh.85@gmail.com…

…….

…….

2014-07-27T11:34:26.45+0530 [App/0]   OUT We are connected to DB
2014-07-27T11:34:26.46+0530 [App/0]   OUT Updated 2 books
2014-07-27T11:34:31.17+0530 [App/0]   OUT We are connected to DB
2014-07-27T11:34:31.17+0530 [App/0]   OUT Updated 2 books
2014-07-27T11:34:31.18+0530 [RTR]     OUT mymongodb01.mybluemix.net - [27/07/2014:06:04:31 +0000] "PUT / HTTP/1.1" 200 1
7 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.153 Safari/537.36" 75
.126.70.43:19986 vcap_request_id:4fa831610b6e79455d0257581a51014e response_time:0.009122768 app_id:ed49744f-1a29-4e3f-9e
1c-8b45e85c3310
2014-07-27T11:35:02.75+0530 [App/0]   ERR
2014-07-27T11:35:02.75+0530 [App/0]   ERR /home/vcap/app/app.js:95
2014-07-27T11:35:02.75+0530 [App/0]   ERR       MongoClient.connect (mongo.url, function(err, db)   {
2014-07-27T11:35:02.75+0530 [App/0]   ERR       ^
2014-07-27T11:35:02.75+0530 [App/0]   ERR ReferenceError: MongoClient is not defined
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at delete_record (/home/vcap/app/app.js:95:2)
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at Server.<anonymous> (/home/vcap/app/app.js:165:12)
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at Server.emit (events.js:98:17)
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at HTTPParser.parser.onIncoming (http.js:2108:12)
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at HTTPParser.parserOnHeadersComplete [as onHeadersComplete] (http.js:121:
23)
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at Socket.socket.ondata (http.js:1966:22)
2014-07-27T11:35:02.75+0530 [App/0]   ERR     at TCP.onread (net.js:527:27)
2014-07-27T11:35:02.80+0530 [RTR]     OUT mymongodb01.mybluemix.net - [27/07/2014:06:05:02 +0000] "DELETE / HTTP/1.1" Mi
ssingResponseStatusCode 0 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.19

 

Happy cooking with Bluemix, MongoDB & Node.js!

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

See also
1. Brewing a potion with Bluemix, PostgreSQL & Node.js in the cloud
2. Spicing up IBM Bluemix with MongoDB and NodeExpress
3. A Cloud Medley with IBM’s Bluemix, Cloudant and Node.js
4. Rock N’ Roll with Bluemix, Cloudant & NodeExpress


Find me on Google+

Brewing a potion with Bluemix, PostgreSQL, Node.js in the cloud

Here is a heady potion made with the key ingredients of IBM’s Bluemix, PostgreSQL & Node.js. In this post I instantiate an app ‘mypgdb01’ on IBM’s Bluemix which uses the services of PostgreSQL in the cloud. This is shown in the picture below.

8

The mypgdb01 is a Webserver running on Node.js in Bluemix and uses the services of PostgreSQL DB. The app mypgdb01 performs basic CRUD (Create, Remove, Update & Delete) on the PostgreSQL.

The code for this can be forked from Devops  from my link mypgdb

The code can also be cloned from GitHub at mypgdb

1) This app uses the Node.js Webstarter kit and the PostgreSQL service.  The Node.js Webserver is created by parsing the VCAP services variable as usual

if (process.env.VCAP_SERVICES) {
var env = JSON.parse(process.env.VCAP_SERVICES);
var credentials = env['postgresql-9.1'][0]['credentials'];
} else {
var credentials = {"uri":"postgre://user:secret1@localhost:5433/db"}
}
var port = (process.env.VCAP_APP_PORT || 1337);
var host = (process.env.VCAP_APP_HOST || '0.0.0.0');
http.createServer(function(req, res) {
console.log("Inside Webserver");
....
....
}).listen(port, host);

The details of VCAP_SERVICES can be see by clicking Runtime for you application and is shown below

9

Note: From above it can be seen that the PostgreSQL DB’s host & port is

"host": "192.155.243.14",
"hostname": "192.155.243.14",
"port": 5433

This different from the host & port of the Node.js Webserver which  can be see in Logs & Files tab (included below) and is

Webserver: host: 0.0.0.0
port:62733

2) Once the Webserver is started the server waits for REST calls of GET,PUT, UPDATE & DELETE as shown below.

// Perform CRUD operations through REST APIs
if(req.method == 'POST') {
insert_records(req,res);
}
else if(req.method == 'GET') {
list_records(req,res);
}
else if(req.method == 'PUT') {
update_record(req,res);
}
else if(req.method == 'DELETE') {
delete_record(req,res);
}

3)  The REST API call  are implements as follows

a) POST:

var insert_records = function(req, res) {
// Connect to DB
var client = new pg.Client(credentials.uri);
client.connect(function(err) {
if (err) {
res.end("Could not connect to postgre: " + err);
}

//Drop table if it exists
client.query("DROP TABLE IF EXISTS emps");
// Creat table and insert 2 records into it
client.query("CREATE TABLE IF NOT EXISTS emps(firstname varchar(64), lastname varchar(64))");
client.query("INSERT INTO emps(firstname, lastname) values($1, $2)", ['Tinniam', 'Ganesh']);
client.query("INSERT INTO emps(firstname, lastname) values($1, $2)", ['Anand', 'Karthik']);

b) GET:

// Select all rows in the table<
var query = client.query("SELECT firstname, lastname FROM emps ORDER BY lastname, firstname");
query.on("row", function (row, result) {
result.addRow(row);
});
query.on("end", function (result) {
// On end JSONify and write the results to console and to HTML output
console.log(JSON.stringify(result.rows, null, "    "));
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write(JSON.stringify(result.rows) + "\n");
res.end();
});

c) UPDATE:

query = client.query("UPDATE emps set firstname = 'Kumar' WHERE firstname='Anand' AND lastname='Karthik'");
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write("Updated record  - Set record with firstname Anand to Kumar\n");

d)DELETE

// Delete the record where the lastname is Karthik
client.query("DELETE FROM  emps WHERE lastname = 'Karthik'");
res.writeHead(200, {'Content-Type': 'text/plain'});
res.write("Deleted record where lastname was Karthik\n");

4) Once the changes are made you can  push the changes to Bluemix using ‘cf’ commands.

The commands are

cf login -a https://api.ng.bluemix.net
cf push mypgdb01-p . -m 512M
cf create-service postgresql 100 pgdb01
cf bind-service mypgdb01 pgdb01

5) In the Bluemix dashboard the app ‘mypgdb01’ should be up and running.

6) To invoke the different database operations we need to make REST API calls to the app.

7) To make the REST API calls you can install Sure Utils -> REST API Chrome extension.  I installed the SureUtils-.REST API Client. This is a Chrome extension and can be installed from Chrome Web Store (search for REST API client). You could choose any REST API client of your choice for the browser you intend to use (Chrome, Firefox)

8) Now we can test the Nodejs-PostgreSQL app with the Sure Utils – Chrome extension

8) The following REST API calls can be made to test the PostgreSQL operations on the database

POST- insert

1

GET – select

2

UPDATE + GET – update + select

The PUT API updates Anand Karthik to Kumar Karthik. This is shown in the GET API call

3

4

.DELETE + GET – delete  + select

5

6

 

Here the DELETE API call deletes the Kumar Karthik record. The GET API call now displays only 1 record.

The console.log output in Bluemix can be see in Files and logs -> stdout.log as shown below

7

The above post shows some basic operations done on a cloud based application that is composed of a Webserver with a PostgreSQL as a backend. The code can be enhanced by adding  front end using Node express.

As mentioned above the code for this can be forked from Devops at mypgdb. The code can also be cloned from GitHub at mypgdb

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

You may also like
1. A Bluemix recipe with MongoDB and Node.js
2. Spicing up IBM Bluemix with MongoDB and NodeExpress
3. A Cloud Medley with IBM’s Bluemix, Cloudant and Node.js
4. Rock N’ Roll with Bluemix, Cloudant & NodeExpress


Find me on Google+

Mixing Twilio with IBM Bluemix

This post walks you through the steps to get started with Twilio on IBM’s Bluemix. Twilio comes as a service that you can add  to your Mobile Cloud or Node.js app. Here’s a quick look at Twilio. Twilio, is a cloud communications IaaS organization which  allows you use standard web languages to build voice, SMS and VOIP applications via a Web API.

Twilio provides the  ability to build VOIP applications using APIs. Twilio itself resides in the cloud and is always available. It also provides SIP integration which means that it can be integrated with Soft switches. Twilio looks really  interesting with its ability to combine the  cloud, Web and VOIP, SMS  and  the like.

This post barely scratches the surface of Twilio & Blue mix. This article provides aa hands-on experience for integration of Twilio with Bluemix and is based on this Twilio blog post. It enables you to send a SMS to your mobile phone by typing in a URL.

As in my earlier post the steps are

1) Fire-up a Node.js  Webstarter application from the  Bluemix dashboard.  In my case I have named the application websms. Once this is up and running

2) Click Add a Service and under ‘Web and Application’ choose Twilio.

3) Enter a  name for the Twilio service. You will also need the Account SID and Authorization token

4) For this go to http://www.twilio.com and sign up2

5) Once you have registered, go to your Dashboard for the Account SID and Auth Token. If the Auth token is encrypted, you can click the ‘lock’ symbol to display the Auth token in plain text.

6) Enter the Accout SID and Auth Token in the Twilio service in Bluemix

7)  To get started you can simply  fork my Twilio  websms code from devops.

8) Now clone the code into a folder you create as follows

git clone https://hub.jazz.net/git/tvganesh/websms

9) You will need to modify the following files

package.json

manifest.yml

app.js

 

10) You can create package.json by running
npm init. Make sure you enter the name of the application you created in Bluemix. In my case it is “websms’ For the rest of the options you can choose the default. Here is the package.json file
"name": "websms",
"version": "0.0.0",
"description": "This README.md file is displayed on your project page. You should edit this \r file to describe your project, including instructions for building and \r running the project, pointers to the license under which you are making the \r project available, and anything else you think would be useful for others to\r know.",
"main": "app.js",
"dependencies": {
"gopher": "^0.0.7",
"express": "^3.12.0",
"twilio": "^1.6.0",
"ejs": "^1.0.0"
},
"devDependencies": {},
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"repository": {
"type": "git",
"url": "https://hub.jazz.net/git/tvganesh/websms"
},
"author": "",
"license": "ISC"
}

11) In the manifest.yml make sure you enter the name of your application and the host

applications:
- host: websms
disk: 1024M
name: websms
command: node app.js
path: .
domain: <your domain>
mem: 128M
instances: 1

12) Lastly make changes to your app.js.

// dependencies
var app = require('gopher'),
twilio = require('twilio');
var config = JSON.parse(process.env.VCAP_SERVICES);
var twilioSid, twilioToken;
config['user-provided'].forEach(function(service) {
if (service.name == 'Twilio') {
twilioSid = service.credentials.accountSID;
twilioToken = service.credentials.authToken;
}
});
// URL test
app.get('/', function(request, response) {
var client = new twilio.RestClient(twilioSid, twilioToken);
client.sendMessage({
to:'<Your mobile number>',
from:'<Number from Twilio dashboard',
body:'Twilio notification through Bluemix!'
}, function(err, message) {
response.send('Message sent! ID: '+message.sid);
});
});

13) After you have made the changes you will need to push the changes to Bluemix using the command line based ‘cf’ tool
14) Login into cf with
cf login – a http://api.ng.bluemix.net

15) Push the websms onto bluemix

16) In the folder where you websms files reside entr the following command
cf push websms -p . -m 512M

17) This should push the code to Bluemix.
Note: If you happen to get a
Server error, status code: 400, error code: 170001, message: Staging error: cannot get instances since staging failed
then you need to make sure to check the changes made to  files app.js, package.,json or the manigfest,yml.

18)  If all things went smoothly, go to your Bluemix dashboard and click the link adjacent to the Routes. You should see that an SMS has been sent as shown

3

19) Your mobile should now display the message that was sent as shown below
Screenshot_2014-06-22-13-41-44

20) Check the  analytics in your Twilio dashboard
1

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+

Test driving Push notification in Bluemix

This post is a continuation of my earlier post ‘Getting started with mobile cloud in Bluemix‘. Here I take a test drive of the push service that Bluemix offers based on the article “Extend an Android app using the Push cloud service” from developerWorks.

This post assumes that you have already completed the changes from my earlier post for the mobile cloud. If you haven’t,  you could clone the code from “mobile data” which is the official IBM version of this app and includes all the changes needed for persisting data in the cloud through their Android.

The Mobile Cloud App I created on Bluemix is “mobtvg“. The main steps to have Push notification service using Bluemix are

  1. GCM services : Get Google API Project number  & GCM API Key
  2. Include the Google Play services library project
  3. Add the jar files to enable Push service
  4. Modify the server side Node.js file to send push notifications to all registered devices
  5. Make necessary code changes
  6. Run the application and test for notification

Here are more details on the above steps

a) GCM services : Google Cloud Messaging for Android (GCM) is a service that allows you to send data from your server to your users’ Android-powered device, and also to receive messages from devices on the same connection. The 1st thing to do is get the Google API Project number & GCM API key.

– Click on Google Developer Console

– Click Create Project. Enter Project name & click Create.

– Note the Project Number on top of the page.

– Click API & Auth on left panel. Click API.

– Scroll down and turn-on Google Messaging for Android

–  Click credentials and click “Create new key”. Click server key. Click create

-Copy API key in the Public API access

Now go the Bluemix dashboard and click your application. Click the Push module. In the Configuration tab, scroll down to Google Cloud Messaging and  click ‘Edit’

Enter the Google API Project Number & GCM API key for both the Sandbox & Production configuration and click Save.

4
b) In Eclipse click Windows->Android SDK manager. Scroll down to the bottom and under Extras select Google Play services. Click install. Once the installation is successful import the project as follows File-Import->Android->Existing Android  code into Workspace. Click Next. In the next screen Browse to the path where your ADT bundle is installed and choose the folder

<ADT-Bundle>\ sdk\extras\google\google_play_services

and Click Ok. Also  check ‘Copy project into workspace’. This will copy.

3

Now build the Google Play Services Project. To do this the project. Click Project->Properties->Android and make sure that you select ‘Is library project’ and then click build.

5

Add a reference to the Google Play services in the Androidmanifest.xml

<meta-data

android:name=”com.google.android.gms.version”

android:value=”@integer/google_play_services_version” />

c) Make all the code changes given in Step 4 of “Extend an Android app using the Push cloud service.

d) In MainActivity.java make sure you change the app_name to the name of your app for e.g

public static final String CLASS_NAME = “MainActivity”;

public static final String APP_NAME = “mobtvg“;

Also ensure that under assets folder you have populated the Application ID in the bluemix.properties file

applicationID=<Application ID from Bluemix>

d) Add ibmcloudcode.jar, ibmpush.jar, android-support-v4.jar (from <Android_SDK_Location>/extras/android/support/v4)

e) Now the Mobile Push project need to include this library project. To do this select your Mobile App project. Click Project->Properties->Android. Click Add and select google-play-services-lib. Note: Make sure “Is library project” is unchecked otherwise you are in for a lot of grief.

8

f) Now you need to make changes to the Node.js application to push any changes from the server to all registered devices.  The code for this is in bluelist-push-node. Note; Making changes through the GUI results in an error that “manifest.yml is not in root node”. So I suggest that you take the ‘cf’ route as follows.

– Clone the code using Git

git clone https://hub.jazz.net/git/mobilecloud/bluelist-push

Go to bluelist-push-node folder

i) Open the app.js with your favorite editor and enter the Application ID of your Bluemix application

//Data Values

var values = {

version:”0.3.1″,

//change this to the actual application id of your mobile backend starter

appID : “<APPLICATION ID>”,

host : “https://mobile.ng.bluemix.net&#8221;

}

ii) Open manifest.yml and change host name & name to the name of your application for e.g.

host: mobtvg

disk: 1024M

name: mobtvg

command: node app.js

path: .

domain: ng.bluemix.net

mem: 128M

instances: 1

iii) Once the changes are complete, open a command propmpt and  login into Bluemix using ‘cf’ as follows

– cd to the directory in which Node.js & manifest.yml exist, Do

cf login – a http://api.ng.bluemix.net

cf push mobtvg -p . -m 512M

(Note the changes are pushed to the mobile cloud app on Bluemix)

This will run through and finally give the status that the app is running successfully.

f) Now that all changes are complete the Mobile Cloud with Push can be tested..

g) Click Window->Android Virtual Device Manager. Click the Device definitions. You choose Google Nexus, Nexus 7. Click Create AVD.

Note: Make sure you choose Google API Level Y and not Android x.x.x API Y.

6

Let the AVD come up and display the current items in the grocery list.

h) Login to Bluemix. Click Push and select the Notifications tab and enter a test message for e.g. “This is a notification from Bluemix” and click send.

7

This will result in a Push Notification to be sent to the AVD. You should see this popup on you AVD as shown below

1

i) Add another AVD through Windows-Android Virtual Device manager. While one AVD is running go to Run->Run Configurations->Target Device and choose the newly created AVD.

j) This will start a second AVD which will refresh with the contents of the grocery list. Now adda new item in one of AVD devices. This will result in a Push notification to the other device that the Bluelist has been updated.

2

There you have it.

1) A mobile cloud applications in which changes persist in the cloud and are refreshed each time the Android device is restarted.

2) A Push notification that is sent to all registered devices whenever there is a change to the list.

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+

Getting started with a Mobile Cloud app with Bluemix

This post gives the key steps to get going in building a Mobile Cloud application on IBM’s Bluemix. This post focuses on using the Android Platform for building the application. IBM Bluemix’es mobile cloud application includes under its hood mobile services like mobile application security, push and mobile data. A Node.js is also thrown in to provide server-side functions.

The Bluemix Mobile architecture is shown below

BuildingMobile

 

As in the previous post an existing Mobile cloud application IBM’s bluelist -base is cloned to get familiarity with the steps involved. The IBM’s bluelist-base app enables the user to maintain a grocery list that persists as mobile data in the cloud instance. To get started perform the following

1) Install ADT + Eclipse bundle from the aforementioned link

2) Unzip and install Eclipse and the ADT bundle

3) Make sure you have the Java JDK for Eclipse. If not install from the following site Java SE Development Kit 8 Downloads

4) Since we will be cloning an existing application and using Eclipse to make the changes we need to install EGit.

5) To do this open Eclipse and select Help-> Install New Software and type in http://download.eclipse.org/egit/updates in the Work with text field and hit enter. You should see the following

1

6) Once EGit is installed the IBM’s bluelist-base App can be cloned as follows

7) In Eclipse click File->Import->Git->Import from Git and click Next

8) Choose Clone URI and Click Next

9) Enter the URI for IBM’s bluelist-base. This shown below

2

10) This will download all the necessary source files and other Android related files and directories into the workspace.

11) After this perform the Steps 2 to Step 6 from the link given Build an Android app using the MobileData cloud service

12) After you make the necessary code changes you are good to go

13) Make sure you right-click and add all the necessary imports required (also Ctrl+Shift + O)

14) Build the Project and make sure that there are no errors

15) You are now ready to run the mobile cloud application. We need to run the mobile app on a Virtual simulator. This can be done as

a) In Eclipse click Window->Android Virtual Device Manager. Click the Device Definitions tab.

b) Choose Nexus 7 (Google) and Click Create AVD.
c) This will open a New Window. Set the following Skin->QVGA and Enter 100 MiB in SD Card size and click OK. This will add this as a AVD.

16) Now run the application.

17) This will bring up the AVD. This takes some time You should see the IBM bluelist showing up as one of the apps.

18) Click on IBM Bluelist. You can add grocery items. These items will persist even if you have to restart your application

3

19) The data is persisted in the IBM’s cloud. This can be checked by logging into BlueMix’es dashboard

4

20) Click the Mobile Data and the data entered in the AVD device will show up in Data Classes drop down.

5

21) The Analytics tab will give a graphical output of the API calls

6

So not the mobile app that is cloud enabled is ready.

Clearly the ability to build Android Apps with the data stored at a cloud opens up numerous possibilities for apps like Evernote, Pocket across several devices.

There you have your first Mobile Cloud App.

Watch this space!

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+

Get your feet wet with IBM Bluemix

This post provides the initial steps to get started on IBM’s Bluemix (currently in beta). Bluemix is open-standard, cloud based Platform-as-a-Service (PaaS) from IBM. Bluemix allows one to quickly put together mobile, web, Big Data, IoT applications. Bluemix is an implementation of IBM’s Open Cloud Architecture, Cloud Foundry which enables developers to rapidly build, deploy, and manage their cloud applications. The developers can tap into a growing ecosystem of available services and runtime frameworks.

Bluemix uses the Softlayer infrastructure to host the user applications. Clients/developers interact with Bluemix either with HTTP or REST as shown below
8

Here are the steps to get going on Bluemix

First things first

I would suggest that you get all the registrations and installations right away.

Bluemix dashboard– Get started by creating an account on Bluemix. This will provide you access to the Bluemix’s dashboard from which you can quickly create applications (mobile, Web, IoT, BigData) etc

Devops: Register for an account with Devops. Devops allows you to easily develop, deploy and track your code online. Devops also allows you to collaborate with others by forking code from their Git repositories

Cf Interface : Install the Command line interface ‘cf” to Bluemix. The ‘cf’ command interface is built with Google’s Go programming language. With ‘cf ‘you can login to Bluemix, create an application, add services and manage your application. You can also do this from the Bluemix’es dashboard.

Install Git: There are multiple ways to develop code for Bluemix. Git command line happens to be one of them, So it makes sense to have this installed. You can install this from this link https://hub.jazz.net/tutorials/clients#installing_git

Install Node.js:The application that this post discusses is based on a Node.js based application so it will help to have it installed. Node.js is a platform that enables building of fast, scalable network applications and created by Ryan Dahl.

Kicking off Bluemix : A good first application to get moving on in Bluemix is the already available Sentiment Analysis of Twitter. This application uses the Node.js ‘sentiment’ module to perform some basic sentiment analysis.

The quickest and most painless way to get started on Bluemix is to ‘fork’ the code for Sentiment Analysis from Devops.

1) Login to your Devops account. Click the following Sentiments link from Devops, in which I have created a slight modification to the sentiment analysis application. You can also clone the code from GitHub at sentiments.

2) Click the Edit Code button at the top. This will open the files and directories in this project (see picture below)

3) Next click the “Fork’ button on the panel on the left side. This will create a copy of the above code in your own repository (see picture below )

1

4) The Twitter sentiment analysis code is in app.js written in Node.js. You can make changes to the code as needed. I have made a few modifications to the code that I had forked. I added changes which adds a textual output of the Twitter sentiment

;

How to make code changes with Web IDE

5) To make code changes double click the app.js file. This will open up the code window. You can use the GUI based IDE to make the code changes and merge with the master branch, The steps are
a) Make the necessary changes and click the symbol shown

2

3. This will open a new window as shown below

3

4) Click the ‘Stage to change’ button.

5) This will move the changes to Staged. Click the ‘Commit’ button and enter the reason for the change and click the “Submit’

4

6) This will move the changes from ‘Staged’ to ‘Commits for master branch’

7) Now click ‘Push all’ and click ‘Ok’ in the Git Push popup window. This will merge the changes into the master branch.

8) Once this done click ‘Build & Deploy’ button

9) Your changes will transition from ‘Pending’ to ‘OK’. Now click the ‘Manage’ button. This will deploy the application with the latest changes on to Bluemix.

10) Do the following to populate the details for the parameters below  with a Twitter app that you create for your application

var tweeter = new twitter({
consumer_key:  <your API key>,
consumer_secret: <your API secret>,
access_token_key:<your access token >,
access_token_secret: <your access token secret>
});

11) To do this log into http://dev.twitter.com

12) Click My applications where your picture is displayed and then click Create application.

13) Enter the details for Name,Description & Website (can be any valid website) and then click Create Twitter application.. This will create the Twitter application.

1

14) Click the API tab. Scroll down to the bottom and click “Create my access token”.

15) This will generate the Access token & Access token secret. Enter all the details (API Key, API secret, Access Token, Access Token secret into app.js and push to the master branch before deploying on Bluemix

 

Code changes with Git command line

11) Incidentally the changes to code can also be made through the Git command shell as follows

a) git clone https://hub.jazz.net/git/tvganesh/sentiments

b) Modify the code using any editor and save the changes

c) Go the directory containing the files and do

git add *

d) git commit -m “Cosmetic” app.js

e) git push

This will push the changes to the git repository in the master branch

8) Click the ‘Build & deploy’ in the top right corner. You should see this

5

9) Click the ‘Manage’ button which will push the application onto the BlueMix

10) To test this application click the link next to ‘Routes’ . Enter a phrase that you would like to search and hit ‘Go’

6

You should see the application checking Twitter periodically for the tweets.

7

Thats it! You have built your first Bluemix application.

The ability to integrate Node.js into your cloud application allows one to easily create powerful applications.

Hasta la vista! I’ll be back!

Disclaimer: This article represents the author’s viewpoint only and doesn’t necessarily represent IBM’s positions, strategies or opinions

Find me on Google+

Latency, throughput implications for the Cloud

The key considerations for any website are latency and throughput. These two parameters are extremely important to web designers as the response time of the web site and the ability to handle large amounts of traffic are directly related to the user experience and the loyalty of returning users.

What are these two parameters and why are they significant? Before looking at latency we need to understand what the response time of the web application is. Ideally this could be defined as the time between the receipt of the HTTP request and the emitting of the corresponding response. Unfortunately any web site hosted on the World Wide Web adds a lot more delay than the response time. This delay comes as the latency of the web site and is primarily due to the propagation and transmission delays on the internet. There are many contributors to this latency starting from the DNS lookup, to the link bandwidth etc.

Throughput on the other hand represents the maximum simultaneous queries or transactions per second that the web application is capable of handling. This is usually measured as transactions-per-second (tps) or queries-per-second (qps).

A good way to understand response time and throughput is to use a oft used example, of a retail store handling customers.  Assuming that there are 5 counter clerks who take 1 minute to check out a customer  we can readily see that as the number of customers to the store increases the throughput increases from 1 customer/minute to a maximum of 5 customers/minute.  Since the cashiers are able to process in 1 minute the response time for the customer is 1 minute/customer. Assuming a 6th customer enters and needs to checkout he/she will have to wait, for e.g.1 minute, if the 5 counter clerks are busy processing 5 other clients,. Hence the response time for the customer will be 1 minute (waiting) + 1 minute (servicing) = 2 minute. The response time increases from 1 minute to 2 minute.  If further clients are ready to check out the length of the wait in the queue will increase and hence the response time. Clearly the throughput cannot increase beyond 5 customers/minute while the response time will increase non-linearly as the clients enter the store faster than they can checked out by the counter clerks.

This is precisely the behavior of web applications. When the traffic to a web site is increased the throughput increases linearly and finally reaches a throughput “plateau”. After this point as the load is increased the throughput remains saturated at this level.  While on the other hand the response time is low at low traffic  it starts to increase non-linearly with increasing load and continues to increase as it maxes out  system resources like the CPU and memory.

When deploying applications on the cloud the latency and throughput are key considerations which are needed to determine the kind of computing resources that  are needed in  the cloud.  Assuming the web application has been optimized and performance tuned for optimum performance what needs to be done is run load testing of the application on the cloud using different CPU instances. For example assume that application is load tested on a small CPU instance.  We need to get the response times and throughput plots with increasing loads. Similarly we now need to deploy the web application on a medium instance and plot response times and the throughput plateaus on the medium instances.

Now the choice as to whether to go for a small CPU instance or medium CPU instance can be calculated as follows. Assuming that the requirements of the web application is to have a response time of ‘t’ seconds then we determine the corresponding traffic handling capacity , for the small CPU instance, say ‘c’ and for the medium CPU instance, let’s assume ‘C’. If the web site has to handle to total traffic of T then we determine the number of instances needed in each case. For the

small CPU instance it will be n= (T/c) + 1

and for

the medium CPU instance it will be N =( T/C)+1.

Now we compute the relative costs of the small and medium CPU instances and identify which is more economical. For example if r1 is the cost per hour of the small CPU instance and R1 is the cost of the medium CPU instance we choose

The small CPU instance if r1 *n < R1 *N (per hour)

While on the other hand if R1 *N < r1 *n then we will choose the medium instance.

Hence the determination of which CPU instance and the configuration of the web application on the cloud will depend on appropriate performance tuning and proper load testing on the cloud. Do also ready my other posts on latency namely ‘The Many faces of latency” and “The Anatomy of Latency“.

Also see latency and throughput in action in the following series of posts

– Bend it like Bluemix, MongoDB with autoscaling – Part 1

– Bend it like Bluemix, MongoDB with autoscaling – Part 2

– Bend it like Bluemix, MongoDB with autoscaling – Part 3

Find me on Google+