Best security in clouds is physical isolation ?

September 8, 2009 in Cloud Computing, Security by jc | No comments

A paper titled “Hey, you, Get Off of My Cloud: Exploring information Leakage in Third-Party Compute Clouds” soon to be released at CCS’09 is exploring the threats resulting from sharing physical compute resources in public clouds.
After demonstrating that despite the likely large number of physical machines in any given public cloud, it is possible to place hostile VMs next to targeted VMs; the authors are listing methods that are taking advantage of information leaking out through shared physical resources.

The paper concludes that the only foolproof solution is to limit sharing with potentially hostile tenants:

A user might insist on using physical machines populated only with their own VMs and, in exchange, bear the opportunity costs of leaving some of these machines under-utilized. For an optimal assignment policy, this additional overhead should never need to exceed the cost of a single physical machine, so large users — consuming the cycles of many servers — would incur only minor penalties as a fraction of their total cost.
Regardless, we believe such an option is the only foolproof solution to this problem and thus is likely to be demanded by customers with strong privacy requirements.

I have one issue with this recommendation: the colocation of many VMs from the same tenant on fewer physical hosts is increasing the risk of having single points of failure. Assuming 8 small instances per physical machine (based on the document estimates), and given the default limit of 20 active VMs per account, most accounts will need less than 3 physical servers, limiting the spread across the availability zones. At that point the tradeoff will be between availability, security and cost.

Compiling rTorrent on openSolaris

August 25, 2009 in OpenSolaris by jc | 7 comments

As I mentioned in a previous post, I’ve recently upgraded a PC to the latest opensolaris release, and had to port some of the applications over. One of these is the fast and efficient rtorrent client. I did not find recent packages in the repositories and had to compile it myself. I found that a future version of opensolaris may have the client integrated, and a case for the SFW consolidation was recently submitted by Huawei Zhang with all the required patches included.

The first step in the install is to make sure that the development environment is setup correctly. From the base opensolaris, I installed the following:

$ pfexec pkg install SUNWncurses
$ pfexec pkg install SUNWcurl
$ pfexec pkg install SUNWgnome-common-devel
$ pfexec pkg install SUNWgmake
$ pfexec pkg install SUNWgcc
$ pfexec pkg install SUNWgnu-automake-110
$ pfexec pkg install SUNWlibtool
$ pfexec pkg install SUNWaconf

The next step is to install libsig++ 2.0 that is required by rlibtorrent. Your mileage may vary, but I had better chance using gmake for all the builds. Note: You will find the lib in the repositories, but I had compilation issues and had to build it myself.

$ wget http://ftp.gnome.org/pub/GNOME/sources/libsigc++/2.0/libsigc++-2.0.18.tar.gz
$ gzip -dc libsigc++-2.0.18.tar.gz | tar xvf -
$ cd libsigc++-2.0.18
$ ./configure
$ gmake
$ pfexec gmake install

 
If you do not change the default location, you should have the libsig++ library installed under /usr/local.
Adding the following will help later to build the rlibtorrent and rtorrent itself.

$ export PKG_CONFIG_PATH=/usr/local/lib/pkgconfig:/usr/lib/pkgconfig

 

Next, on to install rlibtorrent. This is were I would recommend you take to version submitted to SFW with the associated patches.

$ wget http://cr.opensolaris.org/~alz/bittorrent/raw_files/new/usr/src/lib/libtorrent/libtorrent-0.12.2.tar.gz
$ gzip -dc libtorrent-0.12.2.tar.gz | tar xvf -
$ mkdir patches
$ cd patches
$ wget -r -l1 -nd -A.diff http://cr.opensolaris.org/~alz/bittorrent/raw_files/new/usr/src/lib/libtorrent/patches/
$ cd ../libtorrent-0.12.2
$ cat ../patches/rlibtorrent-* | gpatch -p1

 

The following is required because some am files were modified through the patching process.

$ aclocal-1.10 -I./scripts -I.
$ autoheader
$ libtoolize --automake --copy --force
$ automake-1.10
$ autoconf
$ ./configure --enable-shared --disable-static --with-ports --disable-libtool-lock
$ gmake
$ pfexec gmake install

 
Same principles to finally build the rtorrent client.

$ wget http://cr.opensolaris.org/~alz/bittorrent/raw_files/new/usr/src/cmd/rtorrent/rtorrent-0.8.2.tar.gz
$ gzip -dc rtorrent-0.8.2.tar.gz | tar xvf -
$ cd patches
$ wget -r -l1 -nd -A.diff http://cr.opensolaris.org/~alz/bittorrent/raw_files/new/usr/src/cmd/rtorrent/patches/
$ cd rtorrent-0.8.2
$ cat ../patches/rtorrent-0* | gpatch -p1
$ export LDFLAGS='-Wl,-zignore -Wl,-zcombreloc -Wl,-Bdirect  -L/usr/sfw/lib -R/usr/sfw/lib -L/usr/gnu/lib -R/usr/gnu/lib  -L/usr/lib/'
$ export CXXFLAGS=-I/usr/include/ncurses
$ aclocal-1.10 -I./scripts -I.
$ autoheader
$ libtoolize --automake --copy --force
$ automake-1.10
$ autoconf
$ ./configure
$ gmake
$ pfexec gmake install

 
Here it is. Hopefully I did not forget any step or made mistakes while capturing the commands, but you should have enough of a base to start and successfully build rtorrent. Do not hesitate to post a comment with your experience.

Updated on 09/07/2009 to add SUNWlibtool that I forgot. Thanks to Gustavo for pointing it out.

No ZFS in Snow Leopard ?

August 24, 2009 in MacOS, OpenSolaris by jc | No comments

Now that the shipping date for Snow Leopard is approaching, I came to the realization that I will not get rid of my Solaris based NAS. It’s been running flawlessly for the past 2 or 3 years (well, I lost several disks and a controller, but never lost any data), but I was hopping to consolidate everything on a Mac Pro with 8 cores.
Since ZFS is not going to be there, and this apparently until the next major release, I will likely upgrade my PC to a better setup in order to keep running ZFS. Even the open source effort on Mac OS Forge  seems to be going nowhere …

In the mean time, I just upgraded my box to opensolaris 2009.06 and spent some time compiling the tools that I needed on the box, more on that later.

Open source configuration automation update

August 18, 2009 in Cloud Computing, Configuration Management, Data Center Automation by jc | No comments

A while back I listed the open source configuration automation projects: bcfg2, cfengine, puppet and lcfg. Since then, three major things happened:

The puppet community has split

There was a split in the puppet community and a new project saw life as a result: Chef. Chef is describing itself as :

Chef is a systems integration framework, built to bring the benefits of configuration management to your entire infrastructure. With Chef, you can:

  • Manage your servers by writing code, not by running commands. (via Cookbooks)
  • Integrate tightly with your applications, databases, LDAP directories, and more. (via Libraries)
  • Easily configure applications that require knowledge about your entire infrastructure (”What systems are running my application?” “What is the current master database server?”)

More details about the Chef differentiators can be found here.

In a future post, I’ll explore in more details the challenges around configuration automation, and the procedural approach.

Reductive Labs received funding

Reductive Labs, the company responsible for Puppet, has received  $2 Million in funding. Puppet has been gaining traction against cfengine, but it will be interesting to see how Reductive Labs uses its funding, and how the new Chef solution is impacting this progression.

Cloud Computing brought configuration automation in the spotlight

One of the cornerstones of Cloud Computing is the automation of the infrastructure configuration. Either because you want to build a highly automated infrastructure supporting cloud users, or you are putting your application in the cloud. In both cases, infrastructure and applications configuration has to be captured, maintained and automatically provisioned. This will enable rapid scale out, fail over, or in general deployment and redeployment of the managed components.

New

August 12, 2009 in Life by jc | No comments

After a very long hiatus, I feel like I have more to share, and decided to do some summer cleanup on this blog. This post is the first since a long time, many things have changed:

  • I am changing hosting. I was squatting a friend’s collocation, and need to move out. I want to thank Fred for all the resources I used and everything I’ve learned in the process.
  • I’m switching from Pebble to Wordpress. I really liked Pebble, it served me well, I learned a lot, but Java hosting is harder to find. By the way,  I admire Pebble’s author, Simon Brown. If you have a chance, stop by his site. I wish I could move to Jersey too.
  • I’ve changed employer. I’m now working at eBay, as an architect responsible for the datacenter automation area, and the implementation of cloud properties on the site. More on that later.
  • I lost weight, a lot of it (50 pounds), and I’ve  rowed 2 half marathons (indoor), and recently ran one (outdoor). Occasionally, I’ll post about this here too.

I think that’s it.  I hope to be able to post more than I did in the past two years, at least before going through more changes.

Sun to open source Ops Center

December 6, 2007 in Data Centers, Virtualization by admin | No comments

In a previous post, I mentioned the announcement of the Sun’s Ops Center product targeted to the management of virtual environments. In this post, I said that Ops Center was a re-branded N1 System Manager, while in fact, it seems that this is a merge of the Sun Connection and N1 System Manager in one tool :

A highly scalable datacenter automation tool merging discover, update, provisioning, monitoring, and reporting technologies from Sun Connection and N1SM into one tool.

However, by looking at the Oracle World demo, it seems that the UI is radically different from the N1 System Manager (gone the embedded CLI ?).

Also, by looking at the supported platform, it seems that Windows platform is not supported anymore :

Sun N1 System Manager :

From a centralized management console, customers can provision Solaris, Linux, and Windows with a simple drag-and-drop, and monitors the health of systems in an efficient manner.

Sun Ops Center :

The comprehensive, highly scalable Linux and Solaris life cycle management tool.

The good news is that Sun Ops Center will be delivered as open source too :

Building on Sun’s commitment to open standards and customer choice, Sun will continue to innovate the Sun xVM platform and collaborate with open source communities. The first of Sun’s contributions will be the Common Agent Container (CAC) code to the OpenxVM.org community under GPLv3. The CAC is the heart of the management infrastructure for many of Sun’s products, including the Sun xVM Ops Center. In addition, Sun plans to make the entire code base used by Sun xVM Ops Center available to the OpenxVM.org community in the first quarter of 2008.

It’s not clear however if this means the end of life for the N1 System Manager, since right now, the Ops Center does not provide a complete replacement.

 

Sun XVM and Ops Center

October 8, 2007 in Virtualization by admin | No comments

Sun announced new virtualization products last week. It seems to be a version of Xen hypervisor running on solaris . So now, Sun has three Solaris virtualization technologies: XVM, Solaris Containers, and LDOM. So, what it means is that a Sparc Server can be running a solaris container in a solaris XVM in an LDOM. This is 3 layers of virtualizations.

Also, in this announce, Sun presents Ops Center. This is a re-branded N1 System Manager. It seems that the focus will be on the management of the hardware, up to the operating system, combined with Sun Connection.

Mark Hamilton told The Register:

Sun has also announced a new management software package, Ops Center, that will work as a command and control console for physical and virtual gear- that’s to say the hypervisor and Solaris Containers. Sun said the software also includes discovery and inventory, application provisioning, software lifecycles automation, hardware and software monitoring and compliance reporting. Sun brazenly says “it does everything except unpack boxes and rack and cable systems.”

Ops Center is going to be released in December. It will be interesting to see how this can be leveraged by Configuration Automation tools like what BMC announced recently.

 

BMC announce Service Automation

October 8, 2007 in Data Center Automation by admin | No comments

Today, along with the acquisition of Emprisa Networks, and following the acquisition of Realops, BMC announced the Service automation suite of solutions. This includes :

  • Discovery and mapping of the complete IT environment, including its relationships to business processes and transactions
  • Full-scale Configuration Automation capability across network devices, physical and virtual servers, and client systems
  • Change Management to support configuration policy adherence and process automation
  • Orchestration and automation of activities supporting IT processes, reducing errors and expenses in task execution across operational processes and toolsets
  • Automation of Continuous Compliance, including audit and enforcement of standard configurations across heterogeneous IT environments

I’ve been working on the second item in the list for the past year, and now that more of this is announced, I’ll provide more information about what this does, and how runbook automation, coupled with configuration automation can solve many of the challenges that IT operators face today.

 

BMC acquires Network Configuration Automation company Emprisa Networks

October 8, 2007 in Data Center Automation by admin | No comments

Today, BMC is announcing that it acquires Emprisa Network. It will complement the client and server Configuration Automation products, as well as the recent Realops acquisition to provide the Service Automation set of solutions.

Emprisa Networks, Inc., the leading provider of smart network configuration, compliance and change management solutions, offers solutions for extending IT service management initiatives to the network infrastructure. Emprisa’s E-NetAware solution provides immediate return on investment in several key areas – improves service availability with less change errors; automates configuration management for improved operational efficiencies, enhances network security through policy-based configuration management and greatly improves service responsiveness. E-NetAware provides support for routers, switches, firewalls, load balancers, wireless access points and other network devices from over 25 vendors.

I’ll provide more details on this technology in a future blog.

 

Survey of Configuration Management Tools

September 14, 2007 in Configuration Management by jc | No comments

A while back, I tried to list the various open source configuration management tools. I just found an interesting report written by student of the Catholic University of Leuven. In this report Thomas Delaet and Wouter Joosen are reviewing BladeLogic, IBM Tivoli, Opsware, Microsoft SMS and open source projects like Bcfg2, cfengine, LCFG and puppet.

The authors are proposing an interesting taxonomy to evaluate the tools:

1. Abstraction Level: The language used by a configuration management solution can be classified at different levels of abstraction, ranging from high-level end-to-end requirements, to low-level bit-configurations.

2. Specification Language: In this section, we discuss four specification language criteria dealing with usability, domain coverage of a configuration management solution, the grouping mechanism and specifications at multiple abstraction levels.

3. Consistency: In this section, we discuss three criteria that ensure consistency in a computer infrastructure: modeling dependencies, conflict management and workflow management.

4. Distributed Management: Distributed management deals with federated management and distributes translation of configuration specifications.

The conclusion summarize the capabilities of the tools in a nice table.

« Older entries