This presentation traces the evolution of telecom, from the early days of basic of telephony to the current Long Term Evolution(LTE) technology
Tag: technology
Architecting a cloud based IP Multimedia System (IMS)
Here is an idea of mine that has been slow cooking in my head for more than 1 and a 1/2 year. Finally managed to work its way to IP.com. See link below
Architecting a cloud based IP Multimedia System (IMS)
The full article is included below
Abstract
This article describes an innovative technique of “cloudifying” the network elements of the IP Multimedia (IMS) framework in order to take advantage of keys benefits of the cloud like elasticity and the utility style pricing. This approach will provide numerous advantages to the Service Provider like better Return-on-Investment(ROI), reduction in capital expenditure and quicker deployment times, besides offering the end customer benefits like the availability of high speed and imaginative IP multimedia services
Introduction
IP Multimedia Systems (IMS) is the architectural framework proposed by 3GPP body to establish and maintain multimedia sessions using an all IP network. IMS is a grand vision that is access network agnostic, uses an all IP backbone to begin, manage and release multimedia sessions. This is done through network elements called Call Session Control Function (CSCFs), Home Subscriber Systems (HSS) and Application Servers (AS). The CSCFs use SDP over SIP protocol to communicate with other CSCFs and the Application Servers (AS’es). The CSCFs also use DIAMETER to talk to the Home Subscriber System (HSS’es).
Session Initiation Protocol (SIP) is used for signaling between the CSCFs to begin, control and release multi-media sessions and Session Description Protocol (SDP) is used to describe the type of media (voice, video or data). DIAMETER is used by the CSCFs to access the HSS. All these protocols work over IP. The use of an all IP core network for both signaling and transmitting bearer media makes the IMS a very prospective candidate for the cloud system.
This article proposes a novel technique of “cloudifying” the network elements of the IMS framework (CSCFs) in order to take advantage of the cloud technology for an all IP network. Essentially this idea proposes deploying the CSCFs (P-CSCF, I-CSCF, S-CSCF, BGCF) over a public cloud. The HSS and AS’es can be deployed over a private cloud for security reasons. The above network elements either use SIP/SDP over IP or DIAMETER over IP. Hence these network elements can be deployed as instances on the servers in the cloud with NIC cards. Note: This does not include the Media Gateway Control Function (MGCF) and the Media Gate Way (MGW) as they require SS7 interfaces. Since IP is used between the servers in the cloud the network elements can setup, maintain and release SIP calls over the servers of the cloud. Hence the IMS framework can be effectively “cloudified” by adopting a hybrid solution of public cloud for the CSCF entities and the private cloud for the HSS’es and AS’es.
This idea enables the deployment of IMS and the ability for the Operator, Equipment Manufacturer and the customer to quickly reap the benefits of the IMS vision while minimizing the risk of such a deployment.
Summary
IP Multimedia Systems (IMS) has been in the wings for some time. There have been several deployments by the major equipment manufacturers, but IMS is simply not happening. The vision of IMS is truly grandiose. IMS envisages an all-IP core with several servers known as Call Session Control Function (CSCF) participating to setup, maintain and release of multi-media call sessions. The multi-media sessions can be any combination of voice, data and video.
In the 3GPP Release 5 Architecture IMS draws an architecture of Proxy CSCF (P-CSCF), Serving CSCF(S-CSCF), Interrogating CSCF(I-CSCF), and Breakout CSCF(BGCF), Media Gateway Control Function (MGCF), Home Subscriber Server(HSS) and Application Servers (AS) acting in concert in setting up, maintaining and release media sessions. The main protocols used in IMS are SIP/SDP for managing media sessions which could be voice, data or video and DIAMETER to the HSS.
IMS is also access agnostic and is capable of handling landline or wireless calls over multiple devices from the mobile, laptop, PDA, smartphones or tablet PCs. The application possibilities of IMS are endless from video calling, live multi-player games to video chatting and mobile handoffs of calls from mobile phones to laptop. Despite the numerous possibilities IMS has not made prime time.
The technology has not turned into a money spinner for Operators. One of the reasons may be that Operators are averse to investing enormous amounts into new technology and turning their network upside down.
The IMS framework uses CSCFs which work in concert to setup, manage and release multi media sessions. This is done by using SDP over SIP for signaling and media description. Another very prevalent protocol used in IMS is DIAMETER. DIAMETER is the protocol that is used for authorizing, authenticating and accounting of subscribers which are maintained in the Home Subscriber System (HSS). All the above protocols namely SDP/SIP and DIAMETER protocols work over IP which makes the entire IMS framework an excellent candidate for deploying on the cloud.
Benefits
There are 6 key benefits that will accrue directly from the above cloud deployment for the IMS. Such a cloud deployment will
i. Obviate the need for upfront costs for the Operator
ii. The elasticity and utility style pricing of the cloud will have multiple benefits for the Service Provider and customer
iii. Provider quicker ROI for the Service Provider by utilizing a innovative business model of revenue-sharing for the Operator and the equipment manufacturer
iv. Make headway in IP Multimedia Systems
v. Enable users of the IMS to avail of high speed and imaginative new services combining voice, data, video and mobility.
vi. The Service Provider can start with a small deployment and grow as the subscriber base and traffic grows in his network
Also, a cloud deployment of the IMS solution has multiple advantages to all the parties involved namely
a) The Equipment manufacturer
b) The Service Provider
c) The customer
A cloud deployment of IMS will serve to break the inertia that Operators have for deploying new architectures in the network.
a) The Equipment manufactures for e.g. the telecommunication organizations that create the software for the CSCFs can license the applications to the Operators based on innovative business model of revenue sharing with the Operator based on usage
b) The Service Provider or the Operator does away with the Capital Expenditure (CAPEX) involved in buying CSCFs along with the hardware. The cost savings can be passed on to the consumers whose video, data or voice calls will be cheaper. Besides, the absence of CAPEX will provide better margins to the operator. A cloud based IMS will also greatly reduce the complexity of dimensioning a core network. Inaccurate dimensioning can result in either over-provisioning or under-provisioning of the network. Utilizing a cloud for deploying the CSCFs, HSS and AS can obviate the need upfront infrastructure expenses for the Operator. As mentioned above the Service Provider can pay the equipment manufactured based on the number of calls or traffic through the system
c) Lastly the customer stands to gain as the IMS vision truly allows for high speed multimedia sessions with complex interactions like multi-party video conferencing, handoffs from mobile to laptop or vice versa. Besides IMS also allows for whiteboarding and multi-player gaming sessions.
Also the elasticity of the cloud can be taken advantage of by the Operator who can start small and automatically scale as the user base grows.
Description
This article describes a method in which the Call Session Control Function (CSCFs) namely the P-CSCF, S-CSCF,I-CSCF and BGCF can be deployed on a public cloud. This is possible because there are no security risks associated with deploying the CSCFs on the public cloud. Moreover the elasticity and the pay per use of the public cloud are excellent attributes for such a cloudifying process. Similarly the HSS’es and AS’es can be deployed on a private cloud. This is required because the HSS and the AS do have security considerations as they hold important subscriber data like the IMS Public User Identity (IMPU) and the IMS Private User Identity (IMPI). However, the Media Gateway Control Function (MGCF) and Media Gateway (MGW) are not included this architecture as these 2 elements require SS7 interfaces
Using the cloud for deployment can bring in the benefits of zero upfront costs, utility style charging based on usage and the ability to grow or shrink elastically as the call traffic expands or shrinks.
This is shown diagrammatically below where all the IMS network elements are deployed on a cloud.
In Fig 1., all the network elements are shown as being part of a cloud.
Fig 1. Cloudifying the IMS architecture.
Detailed description
This idea requires that the IMS solution be “cloudified “i.e. the P-CSCF, I-CSCF, S-CSCF and the BGCF should be deployed on a public cloud. These CSCFs are used to setup, manage and release calls and the information that is used for the call does not pose any security risk. These network elements use SIP for signaling and SDP over SIP for describing the media sessions. The media sessions can be voice, video or data.
However the HSS and AS which contain the Public User Identity (IMPU) and Private User Identity (IMPI) and other important data can be deployed in a private cloud. Hence the IMS solution needs a hybrid solution that uses both the public and private cloud. Besides the proxy SIP servers, Registrars and redirect SIP servers also can be deployed on the public cloud.
The figure Fig 2. below shows how a hybrid cloud solution can be employed for deploying the IMS framework
Fig 2: Utilizing a hybrid cloud solution for deploying the IMS architecture
The call from a user typically originated from a SIP phone and will initially reach the P-CSCF. After passing through several SIP servers it will reach a I-CSCF. The I-CSCF will use DIAMETER to query the HSS for the correct S-CSCF to handle the call. Once the S-CSCF is identified the I-CSCF then signals the S-CSCF to reach a terminating a P-CSCF and finally the end user on his SIP phone. Since the call uses SDP over SIP we can imagine that the call is handled by P-CSCF, I-CSCF, S-CSCF and BGCF instances on the cloud. Each of the CSCFs will have the necessary stacks for communicating to the next CSCF. The CSCF typically use SIP/SDP over TCP or UDP and finally over IP. Moreover query from the I-CSCF or S-CSCF to the HSS will use DIAMTER over UDP/IP. Since IP is the prevalent technology between servers in the cloud communication between CSCFs is possible.
Methodology
The Call Session Control Functions (CSCFs P-CSCF, I-CSCF, S-CSCF, BGCF) typically handle the setup, maintenance and release of SIP sessions. These CSCFs use either SIP/SDP to communicate to other CSCFs, AS’es or SIP proxies or they use DIAMETER to talk to the HSS. SIP/SDP is used over either the TCP or the UDP protocol.
We can view each of the CSCF, HSS or AS as an application capable of managing SIP or DIAMETER sessions. For this these CSCFs need to maintain different protocol stacks towards other network elements. Since these CSCFs are primarily applications which communicate over IP using protocols over it, it makes eminent sense for deploying these CSCFs over the cloud.
The public cloud contains servers in which instances of applications can run in virtual machines (VMs). These instances can communicate to other instances on other servers using IP. In essence the entire IMS framework can be viewed as CSCF instances which communicate to other CSCF instances, HSS or AS over IP. Hence to setup, maintain and release SIP sessions we can view that instances of P-CSCF, I-CSCF, S-CSCF and B-CSCF executed as separate instances on the servers of a public cloud and communicated using the protocol stacks required for the next network element. The protocol stacks for the different network elements is shown below
The CSCF’s namely the P-CSCF, I-CSCF, S-CSCF & the BGCF all have protocol interfaces that use IP. The detailed protocol stacks for each of these network elements are shown below. Since they communicate over IP the servers need to support 100 Base T Network Interface Cards (NIC) and can typically use RJ-45 connector cables, Hence it is obvious that high performance servers which have 100 Base T NIC cards can be used for hosting the instances of the CSCFs (P-CSCF, I-CSCF, S-CSCF and BGCF). Similarly the private cloud can host the HSS which uses DIAMETER/TCP-SCTP/IP and AS uses SDP/SIP/UDP/IP. Hence these can be deployed on the private cloud.
Network Elements on the Public Cloud
The following network elements will be on the public cloud
a) P-CSCF b) I-CSCF c) S-CSCF d) BGCF
The interfaces of each of the above CSCFs are shown below
a) Proxy CSCF (P-CSCF) interface
As can be seen from above all the interfaces (Gm, Gq, Go and Mw) of the P-CSCF are either UDP/IP or SCTP/TCP/IP.
b) Interrogating CSCF(I- CSCF) interface
As can be seen from above all the interfaces (Cx, Mm and Mw) of the I-CSCF are either UDP/IP or SCTP/TCP/IP.
c) Serving CSCF (S-CSCF) interfaces
The interfaces of the S-CSCF (Mw, Mg, Mi, Mm, ISC and Cx) are all either UDP/IP or SCTP/TCP/IP
d) Breakout CSCF (BGCF) interface
The interfaces of the BGCF (Mi, Mj, Mk) are all UDP/IP.
Network elements on the private cloud
The following network elements will be on the private cloud
a) HSS b) AS
a) Home Subscriber Service (HSS) interface
The HSS interface (Cx) is DIAMETER/SCTP/TCP over IP.
b) Application Server (AS) Interface
The AS interface ISC is SDP/SIP/UDP over IP.
As can be seen the interfaces the different network elements have towards other elements are over either UDP/IP or TCP/IP.
Hence we can readily see that a cloud deployment of the IMS framework is feasible.
Conclusion
Thus it can be seen that a cloud based IMS deployment is feasible given the IP interface of the CSCFs, HSS and AS. Key features of the cloud like elasticity and utility style charging will be make the service attractive to the Service providers. A cloud based IMS deployment is truly a great combination for all parties involved namely the subscriber, the Operator and the equipment manufactures. A cloud based deployment will allow the Operator to start with a small customer base and grow as the service becomes popular. Besides the irresistibility of IMS’ high speed data and video applications are bound to capture the subscribers imagination while proving a lot cheaper.
Also see my post on “Envisioning a Software Defined Ip Multimedia System (SD-IMS)”
Share:
A method for optimal bandwidth usage by auctioning available bandwidth using the OpenFlow protocol
Here is a recent idea of mine that has made it to IP.com (Intellectual Property.com). See link
A method for optimal bandwidth usage by auctioning available bandwidth using the OpenFlow protocol. Here is the full article from IP.com
In this article I provide some more details to my earlier post – Towards an auction-based internet.
Abstract:
As the data that traverses the internet continues to explode exponentially the issue of a huge bandwidth crunch will be a distinct possibility in the not too distant future. This invention describes a novel technique for auctioning the available bandwidth to users based on bid price, quality of service expected and the type of traffic. The method suggested in this invention is to use the OpenFlow protocol to dynamically allocate bandwidth to users for different flows over a virtualized network infrastructure.
Introduction:
Powerful smartphones, bandwidth-hungry applications, content-rich applications, and increasing user awareness, have together resulted in a virtual explosion of mobile broadband and data usage. There are 2 key drivers behind this phenomenal growth in mobile data. One is the explosion of devices viz. smartphones, tablet PCs, e-readers, laptops with wireless access. The second is video. Over 30% of overall mobile data traffic is video streaming, which is extremely bandwidth hungry. Besides these, new technologies like the “Internet of Things” & “Smart Grids” now have millions and millions of sensors and actuators connected to the internet and contending for scarce bandwidth. In other words there is an enormous data overload happening in the networks of today.
Two key issues of today’s computing infrastructure deal with data latency and the economics of data transfer. Jim Gray (Turing award in 1998) in his paper on “Distributed Computing Economics” tells us that the economics of today’s computing depends on four factors namely computation, networking, database storage and database access. He then equates $1 as follows
One dollar equates to
= 1 $
≈ 1 GB sent over the WAN
≈ 10 Tops (tera CPU operations)
≈ 8 hours of CPU time
≈ 1 GB disk space
≈ 10 M database accesses
≈ 10 TB of disk bandwidth
≈ 10 TB of LAN bandwidth
As can be seen from above breakup, there is a disproportionate contribution by the WAN bandwidth in comparison to the others. In others words while the processing power of CPUs and the storage capacities have multiplied accompanied by dropping prices, the cost of bandwidth has been high. Moreover the available bandwidth is insufficient to handle the explosion of data traffic.
It is claimed that the “cheapest and fastest way to move a Terabyte cross country is sneakernet (i.e. the transfer of electronic information, especially computer files, by physically carrying removable media such as magnetic tape, compact discs, DVDs, USB flash drives, or external drives from one computer to another).
While there has been a tremendous advancement in CPU processing power (CPU horsepower in the range of petaflops) and enormous increases in storage capacity(of the order of petabytes) coupled with dropping prices, there has been no corresponding drop in bandwidth prices in relation to the bandwidth capacity.
It is in this context an auction-based internet makes eminent sense. An auction-based internet would be a business model in which bandwidth would be allocated to different data traffic on the internet based on dynamic bidding by different network elements. Such an approach becomes imperative while considering the economics and latencies involved in data transfer and the emergence of the promising technology known as the OpenFlow protocol. This is further elaborated below
Description
As mentioned in Jim Turing’s paper a key issue that we are going to face in the future has to do with the economics of data transfer and the associated WAN latencies
As can be seen there are 3 distinct issues with the current state of technology
1) There is an exponential increase in data traffic circling the internet. According to a Cisco report the projected increase in data traffic between 2014 and 2015 is of the order of 200 exabytes (10^18)).The internet is thus clogged due to the many bandwidth hungry applications and millions of devices that make the internet
2) WAN latencies and the economics of data transfers are two key issues of the net
3) Service Providers have not found a good way to monetize this data explosion.
Clearly bandwidth is a resource that needs to be utilized judiciously given that there are several contenders for the usage of bandwidth.
Detailed description: This invention suggests a scheme by which internet bandwidth can be auctioned between users based on their bid price, Quality of Service (QoS) required and the type of traffic (video, voice, data, streaming). The energy utility already auctions electricity to the highest bidder. This invention suggests a similar approach to auction scarce bandwidth to competing bidders.
The internet pipes get crowded at different periods of the day, during seasons and during popular sporting events. This invention suggests the need for an intelligent network to price data transfer rates differently depending on the time of the day, the type of traffic and the quality of service required. In this scheme of things the internet will be based on an auction mechanism in which different devices bid for scarce bandwidth based on the urgency, speed and quality of services required.
Such a network can be realized today provided the network and the network elements that constitute the internet implement the OpenFlow protocol.
Software Defined Networks (SDNs) is the new, path breaking innovation in which network traffic can be controlled programmatically through the use of the OpenFlow protocol. SDN is the result of pioneering effort by Stanford University and University of California, Berkeley and is based on the Open Flow Protocol and represents a paradigm shift to the way networking elements operate.
SDNs can be made to dynamically route traffic flows based on decisions in real time. The flow of data packets through the network can be controlled in a programmatic manner through the OpenFlow protocol. In order to dynamically allocate smaller or fatter pipes for different flows, it necessary for the logic in the Flow Controller to be updated dynamically based on the bid price, QoS parameters and the traffic type.
The OpenFlow protocol has a Flow Controller element which can be made to create different flows by manipulating the flow tables of the different network elements. Hence the Flow Controller depending on the bid price, the bandwidth rate and the QoS will auction the different bids and create different flows for different users. The Flow Controller will then update the flow tables of the network elements that will participate to realize this end-to-end flow of traffic for different users.
A typical scenario can be visualized as below
In the above figure different users bid for available bandwidth. For e.g. User A could bid for A Mbps @ $a/bit for traffic type A, User B could bid for B Mbps @ $b/bit for traffic type B and User C could bid for C Mbps @ $c/bit for traffic type C. The different QoS parameters like delay, throughput, and jitter are all sent in the user requests. The Flow controller receives all these bids with associated parameters and auctions the available bandwidth against the bid prices that the network elements bid for. The Flow Controller then ranks the bids against the most optimal bandwidth allocation that has the highest return.
The Flow Controller can then allocate different bandwidths to the different users based on the bids from the highest to the lowest, quality of service and the type of traffic. Software Defined Networks (SDNs) can then create different flows for across the networks. SDN can create different slices of network elements from end-to–end for each of the different flow requirements.
The Flow Controller can then create these flows and update the flow tables of the network elements based on the allotted speeds for the bid price.
This is shown diagrammatically below
For e.g. we could assume that a corporate has 3 different flows namely, Immediate, ASAP (As soon as possible) and price below $x. Based on the upper ceiling for the bid price, the OpenFlow controller will allocate a flow for the immediate traffic of the corporation. For the ASAP flow, the corporate would have requested that the flow be arranged when the bid price falls between a range $a – $b. The OpenFlow Controller will ensure that it can arrange for such a flow. The last type of traffic will be allotted a default flow during non-peak hours. This will require that the OpenFlow controller be able to allocate different flows dynamically based on winning the auction process that happens in this scheme.
Using the OpenFlow paradigm to auction bandwidth
These will be the typical steps that will occur during
- Let us assume that it is the period of the day when the usage is at its peak
- Let there be 3 users User A, User B and User C who would like to video-conference, video stream and make a voice call respectively
- Depending on the urgency and the price that the users can afford these 3 users will bid for a slice of a bandwidth to complete their call
- Let user A request A Mbps @ $a/bit for QoS parameters p(a). Let user B request B Mbps @ $b/bit for QoS parameters p(b) and user C request C Mbps @ $c/bit for QoS parameters p(c).
- When the Flow Controller receives these requests, based on the available bandwidth at its disposal (assuming it has already used X Mbps for already existing flows) it will normalize these requests and auction them so that it results in the highest bid winning its requested bandwidth slice followed by the ones lower than it. If a user does not qualify the auction the user will have a bid at a later time according to some algorithm. Let us assume that user A and user C win their bids
- The Flow Controller will now algorithmically decide the contents of the flow tables of the intervening network elements and will accordingly populate these flow tables
- The flows for User A and User C are now in progress.
- The Flow Controller will accept bids whenever there is spare bandwidth that can be put up for auction.
As can be seen such a mechanism will result in a varying price for bandwidth with the highest value during peak periods and lower values during off-peak periods.
Benefits: The current protocols of the internet of today namely IntServ, DiffServ allocate pipes based on the traffic type & class which is static once allocated. This strategy enables OpenFlow to dynamically adjust the traffic flows based on the current bid price prevailing in that part of the network. Moreover the usage of OpenFlow protocol can generate a lot more granualar flow types.
The ability of the OpenFlow protocol to be able to dynamically allocate different flows will once and for all solve the problem of being able to monetize mobile and fixed line data This will be a win-win for both the Service Providers and the consumer. The Service Provider will be able to get a ROI for the infrastructure based on the traffic flowing through his network. Users can decide the type of service they are interested and choose appropriately. The consumer rather than paying a fixed access charge could have a smaller charge because of low bandwidth usage.
Conclusion: An auction-based internet is a worthwhile business model to pursue. The ability to route traffic dynamically based on an auction mechanism in the internet enables the internet infrastructure to be utilized optimally. It will serve the dual purpose of solving traffic congestion, as highest bidders will get the pipe but will also monetize data traffic based on its importance to the end user.
Share:
The computer is not a dumb machine!
“The computer is a dumb machine. It needs to be told what to do at every step”. How often have we heard of this refrain from friends and those who have only an incidental interaction with computers? To them a computer is like a ball which has to be kicked from place to place. These people are either ignorant of computers, say it by force of habit or have a fear of computers. However this is so far from the truth. In this post, my 100th, I come to the defense of the computer in a slightly philosophical way.
The computer is truly a marvel of technology. The computer truly embodies untapped intelligence. In my opinion even a safety pin is frozen intelligence. From a piece of metal the safety pin can now hold things together while pinning them, besides incorporating an aspect of safety.
Stating that the computer is a dumb machine is like saying that a television is dumb and an airplane is dumber. An airplane probably represents a modern miracle in which the laws of flight are built into every nut and bolt that goes into the plane. The electronics and the controls enable it to lift off, fly and land with precision and perform a miracle in every flight.
Similarly a computer from the bare hardware to the upper most layer of software is nothing but layer and layer of human ingenuity, creativity and innovation. At the bare metal the hardware of the computer is made up integrated chips that work at the rate of 1 billion+ instructions per second. The circuits are organized so precisely that they are able to work together and produce a coherent output, all at blazing speeds of less than a billionth of a second.
On top of the bare bones hardware we have some programs that work at the assembly and machine code made of 0’s and 1’s. The machine code is nothing more than an amorphous strings of 0’s and 1’s. At this level the thing that is worked on (object) and the thing that works on it (subject) are indistinguishable. There is no subject and object at this level. What distinguishes then is the context.
Over this layer we have the Operating System (OS) which I would like to refer to as the mind of the computer. The OS is managing many things all at once much like the mind has complete control over sense organs which receive external input. So the OS manages processes, memory, devices and CPU (resources)
As humans, we like to pride ourselves that we have consciousness. Rather than going into any metaphysical discussion on what consciousness is or isn’t it is clear that the OS keeps the computer completely conscious of the state of all its resources. So just like we react to data received through our sense organs the computer reacts to input received through devices (mouse, keyboard) or its memory etc. So does the computer have consciousness?
You say human beings are capable of thought. So what is thought but some sensible evaluation of known concepts? In a way the OS is also constant churning in the background trying to make sense of the state of the CPU, the memory or the disk.
Not to give in I can hear you say “But human beings understand choice”. Really! So here is my program for a human being
If provoked
Get angry
If insulted
Get hurt
If ego stoked
Go mad with joy
Just kidding! Anyway the recent advances in cognitive computing now show it is possible to have computers choose the best alternative. IBM’s Watson is capable of evaluation alternative choices.
Over the OS we have compilers and above that we have several applications.
The computer truly represents layers and layers of solidified human thought. Whether it is the precise hardware circuitry, the OS, compilers, or any application they are all result of human thought and they are constantly working in the computer.
So if your initial attempt to perform something useful did not quite work out, you must understand that you are working with decades of human thought embodied in the computer. So your instructions should be precise and logical. Otherwise your attempts will be thwarted.
So whether it’s the computer, the mobile or your car, we should look and appreciate the deep beauty that resides in these modern conveniences, gadgets or machinery.
Share:
Adventures in LogParser, HTA and charts
In my earlier post “Slicing and dicing with LogParser & VBA” I had mentioned that LogParser is really a slick Microsoft utility that can be used to obtain information on files, event logs, IIS logs etc. Continuing on the journey in LogParser I came to realize that you can also create cool charts with output of LogParser which can be either a line graph, a pie chart , a 3D pie chart a 3D bar chart etc. The options are many. So I started to play around with the utility.
To create a chart you can run the command from a LogParser prompt. Some samples are shown below
LogParser “SELECT TOP 10 TO_LOWERCASE (Name) AS NewName, Size INTO .\chart.gif, Path, LastWriteTime FROM ‘” & files & “‘ WHERE NOT Attributes LIKE ‘%D%’ AND NOT ATTRIBUTES LIKE ‘%H%’ ORDER BY Size DESC ” & “-chartType:column3D -i:FS –chartTitle: “My chart”
This will create a gif file with a 3D bar chart with the top 10 files by size
Similarly you could also create a 3D pie chart as follows
LogParser “SELECT TOP 5 Name, Size INTO c:\Chart.gif FROM C:\*.* ORDER BY Size DESC” -chartType:PieExploded3D -i:FS
However I wanted to create these charts in a HTA application and display it dynamically along with the output of LogParser. Thankfully the procedure is very similar. Here is what you need to do for this.
To do this you need to set up the environment as below. I have used VBscript.
Set objLogParser = CreateObject(“MSUtil.LogQuery”)
Set objInputFormat = CreateObject(“MSUtil.LogQuery.FileSystemInputFormat”)
Then you need to specify the chart options
Set objOutputChartFormat = CreateObject(“MSUtil.LogQuery.ChartOutputFormat”)
objOutputChartFormat.groupSize = “400×300”
objOutputChartFormat.fileType = “GIF”
objOutputChartFormat.chartType = “Column3D”
objOutputChartFormat.categories = “ON”
objOutputChartFormat.values = “ON”
objOutputChartFormat.legend = “ON”
Finally create a LogParser query and execute it as shown below where “topN” & the directory path “files” is taken as input from the user
strQuery1 = “SELECT TOP ” & topN & ” Name, Size INTO c:\tes\filesize.gif FROM ” & files & ” WHERE NOT Attributes LIKE ‘%D%’ AND NOT ATTRIBUTES LIKE ‘%H%’ ORDER BY Size DESC ”
objOutputChartFormat.config = “c:\test\FileSize.js”
Set objRecordSet1 = objLogParser.ExecuteBatch(strQuery1, objInputFormat , objOutputChartFormat )
To specify the chart title, the X axis & Y axis a javascript/VBscript file has to be created as below (FileSize.js) which is specified in objOutputChartFormat.config
FileSize.js (contents)
// Set the title above the chart.
chart.HasTitle = true;
chart.Title.Caption = “Top N files by size”
// Set the border style for the chart.
chartSpace.Border.Color = “#000000”;
chartSpace.Border.Weight = 2;
// Change the background color for the plot area.
chart.PlotArea.Interior.Color = “#f0f0f0”;
// Set the font size for the chart values.
chart.SeriesCollection(0).DataLabelsCollection(0).Font.Size = 6;
// Set the caption below the chart.
chartSpace.HasChartSpaceTitle = true;
chartSpace.ChartSpaceTitle.Caption =
“This chart shows the Top N files by file sizes in the specified directory “;
chartSpace.ChartSpaceTitle.Font.Size = 10;
chartSpace.ChartSpaceTitle.Position = chartSpace.Constants.chTitlePositionBottom;
// Set the style and caption for the Y axis.
chart.Axes(0).Font.Size = 8;
chart.Axes(0).HasTitle = true;
chart.Axes(0).Title.Caption = “File Name”;
chart.Axes(0).Title.Font.Size = 9;
// Set the style and caption for the X axis.
chart.Axes(1).Font.Size = 7;
chart.Axes(1).HasTitle = true;
chart.Axes(1).Title.Caption = “Size in bytes”;
chart.Axes(1).Title.Font.Size = 9;
Lastly to display the chart dynamically as it is created in the HTA file do the following
imagearea.innerHTML = “
”
where imagearea will be specified in the HTML portion as
where filesize.png is any image prior to the creation of the chart through LogParser.
A sample output is shown below
LogParser charts are really cool and well worth the effort!
Also see
Brewing a potion with Bluemix, PostgreSQL, Node.js in the cloud
A Bluemix recipe with MongoDB and Node.js A Cloud medley with IBM Bluemix, Cloudant DB and Node.js
Rock N’ Roll with Bluemix, Cloudant & NodeExpress
You may also like
– A crime map of India in R: Crimes against women
– What’s up Watson? Using IBM Watson’s QAAPI with Bluemix, NodeExpress – Part 1
– Bend it like Bluemix, MongoDB with autoscaling – Part 1
– Analyzing cricket’s batting legends – Through the mirage with R
– Masters of spin: Unraveling the web with R
Share:
Windows Resource Management Tool – Technology choices
There are the following technology choices for a Windows Automation Tool
a) VBA with Excel
b) Perl
c) Powershell
d) HTML for Applications (HTA)
1. VBA with Excel
VBA (Visual Basic for Applications) with Excel is good option to consider. VBA allows for a quick development of a fairly reasonable user interface. VBA includes a Userform to which one can controls like textbox, listbox, combo box, radio button etc. These controls can then invoke VBscripts in the background which can perform resource management tasks. For some more detail on how to create the Userform with controls look at my earlier post Stir fry a VBA application quickly. Besides, the results can be populated in the Excel sheet
A screen shot of a VBA with Excel is shown below
Pros
- Ease of building a GUI using the Visual Basic User form with appropriate controls
- The VBA will include VBscripts that will perform resource management tasks
- Can populate the results into Excel sheet.
- The results from the Excel sheet can be saved as CSV, HTML etc
Cons
- Not all client sites have MS Office installed on them. This makes it difficult to deploy Excel with VBA.
2. Perl
Perl allows for easy creation of scripts to manage Windows Resources. The language is easy to use and facilitates quick prototyping. However Perl by itself does not provide for the creation of a GUI. Perl scripts can only be run in command line mode.
Pros
- Ease of scripting
Cons
- Does not allow for creation of a GUI
3. Powershell
Powershell is a convenient way for creating resource management scripts. It is surprising the Microsoft took such a long time to progress from the clunky “command” shell to Powershell which is more on the lines of Korn, Bourne and Unix shell. The nice part is that instead of the cryptic Unix commands the commands have a verb-noun combination for e.g. get-date, get-command etc. The benefit of Powershell is the extensive help that is available for each command. Powershell is in many ways similar to Perl. Building a GUI with Powershell is quite involved and requires quite a bit of programming. Moreover Powershell is installed by default only on Windows 7 and later. For an excellent tutorial on Powershell do read Powershell tutorial by Jesse Hamrick
Pros
- Ease of scripting for resource management
- Extensive help available for query while building resource management commands
- Can format output easily into list, table or Excel sheet
- Easy to build stand-alone scripts
Cons
a. Building a GUI based tool is fairly involved
4. HTML Applications (HTA)
HTML application brings the power of the browser for the front end with VBscript or JScript as the backend. But to build a easy to use GUI there is a need to use Web page builders like Microsoft’s FrontPage, Dreamweaver etc. The controls on the Web page can invoke VBScript to run resource management tasks
Pros
- Can build a user-friendly front-end using Web page builder tools
- Resource management tasks can be executed by VBscript
Cons
- Needs a regular Web page building tool like MS FrontPage or Dreamweaver
Share:
Stacks of protocol stacks
Communication protocols like any other technology arrive on the scene to solve a particular problem. Some protocols endure while many perish. The last 60 years or so have seen a true proliferation of protocols in various domains.
So what is a protocol?
In my opinion a protocol is any pre-defined set of communication rules.
For e.g. consider the exchange between me and you
Me: “Thank You”
You: “You’re welcome”.
A more complex exchange could be
You: “How are you doing today?”
Me:”Fine. And yourself?”
You: “Great”
These are “protocols of courtesy or decorum”. There are many such protocols in daily use so there is little wonder that the technological world is full of protocols.
A couple of decades back there were 3 main standard bodies that came up with protocols namely IEEE (for LANs), IETF for the internet and ITU-T for telecom. Now there are many more bodies for e.g. CableLabs for cable television, WiMAX forum for WiMAX, NFC Forum etc.
Also protocols exist both for wired and the wireless domain. The protocols differ based on the distance for which the protocol will apply. This post will try to take a look at the some of most important in this. Certainly many will slip through the cracks, so beware!
Near Field Communication (NFC): This is a wireless protocol of the order of a few centimeters primarily for contactless data transfers. Its primary use is for mobile payment. As opposed to Bluetooth there will be no necessity for device pairing. The NFC standards are maintained by the NFC Forum.
Bluetooth: This is another wireless protocol and uses the 2.4- 2.48 GHz band for data exchange and is commonly used in mobile phones, TVs, and other devices. This protocol requires pairing of devices prior to data transfer. The Bluetooth details are maintained in Bluetooth Special Interest Group.
Zigbee: Zigbee is a low powered, low cost wireless protocol that will connect devices within residential homes. Zigbee has a data rate of 250 kbps and is based on the IEEE 802 standard for Personal Area Network (PAN) or Home Area Network (HAN). Zigbee will be protocol of choice in the Smart Home which will be part of Smart Grid concept. More details can be found at the Zigbee Alliance.
LAN protocols: LAN protocols are wired protocols. The main 3 LAN protocols are IEEE 802.3 (Ethernet), IEEE 802.4 (Token Bus) & IEEE (Token Ring) are used in enterprises, schools or small buildings of the order of a few 100 meters. LAN protocols ensure transmission speeds of the order of 10 Mbps – 40 Mbps.
WiFi: WiFi provides wireless access in residential homes, airports, cafes at a distance of 20 meters with speeds of 2 Mbps – 8 Mbps (802.11a/b/e/g). Wireless hotspots use WiFi protocols
Super WiFi/Whitespaces: Whitespaces refers to using abandoned TV frequency bands for wireless data transmission around the 700 MHz range. Whitespaces can travel larger distances typically around 100 km and through trees and walls. This is nascent technology and is based on IEEE 802.22 protocol. A new forum for taking this technology forward is the Whitespace Alliance.
Telecom protocols
ISDN: This protocol is governed by the Q.931 standards and was supposed to carry high speed data (64 kbps???) from residential homes, This protocol went into relative obscurity soon.
Wired Trunk protocols: There are several trunk protocols that connect digital exchanges (digital switches) for e.g. ISUP (Q.763), BTUP, TUP. These protocols exchange messages between central offices and are used for setting up, maintaining and release of STD voice calls.
Internet Protocols
The predominant protocol of the internet is TCP/IP (RFC 793). There are several other protocols that work in the internet. A few of them
Exterior Gateway Protocol (EGP)
OSPF Open Shortest Path First protocol
Interior Gateway Protocol (IGP)
RSVP & DiffServ
WAN protocols: There is a variety of protocols to handle communication between regions or across a large metropolitan area. The most common among these are
MPLS: Multi-protocol Label System.
ATM : Asynchronous Transfer Mode
Frame relay:
X.25:
Protocols that are exist in both the Internet & Telecom domain
A number of protocols work in concert to setup, maintain and release multi-media sessions
SIP/SDP: Session Initiation Protocol (RFC 3261 et al) /Session Description Protocol (RFC 2327)
SCTP/RTP/RTSP: Session Control Transport Protocol/Real Time Protocol/Real Time Secure Protocol – These protocols are used to send and control media packets.
MGCP/Megaco: This is a protocol used to control the Softswitch.or the Media Gateway Controller (MGC)
WiMAX: (Worldwide Interoperability for Microwave Access) is a technology for wirelessly delivering high-speed Internet service to large geographical areas. WiMAX offers data speeds in the range of 40 Mbps – 70 Mbps. This is an IEEE 802.16 family of protocols. Details about WiMAX can be obtained at WiMAX Forum.
DOCSIS: DOCSIS is the protocol that is used in cable TV and uses hybrid fiber co-axial cables for transmission. This protocol is also used these days for internet access. More details regarding the DOCSIS protocol can be found at CableLabs.
Note: I will be adding more substance and body to this post soon …
Giga thoughts ... 