Last we spoke, it was about network deviceconfiguration management. Let's move our focus up the stack toapplications and management of their configuration. Whether enterprise or cloud-architected, running on physical servers, in virtual machines or in containers, how are you managing your applications?

Puppet, Chef, Ansible and Salt are popular answers to this question and leading contenders for initial provisioning and management of configuration drift of data center applications -whether they be common off the shelf (COTS) or custom built applications. Two of these configuration management technologies, Puppet and Chef, are supported by Cisco Intelligent Automation for Cloud 4.1. The collection of features enveloping these two Ruby-based technologies within Cisco IAC is referred to as Application Configuration Management (ACM).

Approach to Agent Bootstrapping

Puppet and Chef are similar in nature -in more ways that we'll discuss in this post. An example of similarity being that both of these ACM technologies require an agent (Puppet) or client (Chef) installed on the server under management (node).

Agent Bootstrapping Methods

Both types of ACM technologies support client-only and client/server deployment models, referred to as agent/master for Puppet and client/server for Chef installations. Whether only using an agent-only (client-solo -Chef) or using an agent/master deployment model, unless your virtual or physical server image has the agent preinstalled, you'll need to go perform the prerequisite work of agent installation.

IAC performs this dirty work by bootstrapping the appropriate agent (or client) whether on initial server provisioning or on-demand on any existing server when a user assigns an application to a server. Mechanics used to perform agent installation varies. The mechanics used within IAC are listed in the "Agent Bootstrapping Methods" chart. Initially, IAC used WinRM as its mechanism to bootstrap agents on Windows severs until customer feedback drove use of an alternative mechanism -psexec. We found that customer security teams were either uncomfortable with or had policy in place against the use of WinRM as a method to execute scripts remotely and made the switch to psexec, which "is a light-weight telnet-replacement that lets you execute processes on other systems, complete with full interactivity for console applications, without having to manually install client software".

Part of the agent installation involves establishing a connection between the ACM server (Puppet Master or Chef Server) and the node (server with agent/client installed). IAC orchestrates the registration of the node with it's respectively, assigned ACM server. This process is different depending on whether Puppet or Chef is used.  In the case of Chef, IAC has the chef-client register with the Chef server using the private key assigned to the chef-validator, which IAC loads into the node during client installation. In the case of Puppet, IAC performs an initial puppet agent run, which lodges a certificate authorization request on the Master, which IAC subsequently orchestrates the signing of on the Master. With agent bootstrap complete and authorized, secure communication between the ACM server and client, attention is turned to the management of connections IAC may have established with n number of Puppet or Chef servers.

System Health -ACM

Connection and System Health

In the case of client/server deployments, IAC will establish connection to one or more Puppet Masters and one or more Chef Servers. Each connection is treated with care as the health of each connection facilitates IAC's ability to successfully orchestrate applications. Connections are established using a service account permissioned appropriately. The health of the connection between each ACM server is evaluated once every 30 minutes by default. Connection health is determined by performing connectivity, authentication and authorization tests. Details of these tests and a screenshot of the System Health console can be seen in the  "System Health -ACM" chart.

CloudSync Finite State Machine

Cloud Object Model and CloudSync

Immediately after establishing a healthy connection, CloudSync runs. CloudSync is a synchronization process driven by a finite state machine whose responsibility is to not only perform initial object discovery and granular fingerprinting -essentially a deep interrogation of cloud objects and their attributes -but also, manage ongoing reconciliation of infrastructure changes with respect to their representation of the provider's cloud infrastructure as modeled within the service catalog. Note the "CloudSync Finite State Machine" chart, which is laced with Extension Points, where cloud administrators may insert custom logic on state transition for any given object within the model. Once collected, this inventory (e.g. a Chef Role) is presented to the cloud administrator for  the ACM server for use within their cloud. Cloud administrators may choose to register the discovered objects for use by end users.

Cisco IAC Cloud Object Model -Chef

Cisco IAC Cloud Object Model -Puppet

For example, the cloud administrator may choose to register a Puppet Role as being available for end users to assign to a server. Registration of this role may include assignment of additional metadata, including price of the role as a one-time or recurring charge for use of the application and assignment of tenant permissions (whether to make the role available to all tenants or only select tenant(s)).

It's through the relationships derived within the Cloud Object Model and assignment of tenant permissions that the specific applications are presented to a given end user. Service Resource Containers are used as a logical construct owned by the cloud administrator wherein tenant-specific resources may be hosted. Applications delivered to tenants may be created in a virtual data center that is serviced by either a Puppet Master or Chef Server. See the Cisco IAC documentation for further details on other constructs within these and other models.

Manage Applications -Node Classification

Approach to Node Classification

Once registered for use, applications become visible to end users, who may assign applications to their servers whether during initial server provisioning or to an existing server. Upon selection of application(s) by the end user, IAC classifies the node by writing a hiera file (Puppet) or by writing a run-list (Chef) on the respective ACM server and forces an immediate agent run to ensure application configuration is promptly enforced.

In this sense, IAC provides a common user experience for node classification irrespective of the underlying technology chosen by the cloud provider (the organization running and administering IAC). As the IAC product suite evolves, so has our approach in terms of classification via Puppet and the more programmatically effective use of a custom-written External Node Classifier, taking advantage of the ability for the node_terminus configuration to to interact with an ENC.

Application Configuration Management Highlights

CloudSync'ed Applications

• Integration with Puppet and Chef
  • Connections ton number of servers
  • System health checks for these servers
  • Application infrastructure discovery (CloudSync)
  • Bootstrapping of agents (green and brownfield)
• Financial Management
  • Pricing of applications
  • Showback for application orders
  • Run rates including application consumption by user, org, tenant
• Multi-tenancy Financial Management -Application Pricing & Showback
  • Tenant-specific application catalogs
  • Tenant/application consumption dashboards
• Provisioning
  • Application provisioning for virtual machines, physical servers
  • "My Applications" interface for application management
• Service Offering Elections
  • 3-tiers of control on enable/disable application configuration management services at provider, tenant and organization levels
• Multi-Cloud Platform Support
  • Support same services ubiquitously across all platforms
  Financial Management -Application Run Rates
• Application User Persona
  • "My Application" interface for application management
  • ACM Server and application usage dashboard

Cognizant of the plethora of application configuration management tools available to Cisco customers, including commercial, open source, and homegrown tools, we're very interested to hear which ones you have found to be the best fit in your environment. Have you established revision control practices as you manage infrastructure as code?Having reviewed Cisco's approach within its cloud management platform, IAC, whether you manage configuration of physical servers, virtual machines or use CM to build containers or hosts that run containers, how does your approach compare?