Saturday, December 17, 2016

What is an API for non-technical people

During conversations over the last few weeks, I have been asked more than once by non-technical people... "What is an API ?"

As technology evolves to have more interconnected systems, many will find the term API becoming a larger part of their lives. This includes both business people and technical people alike.

I thought I would share an analogy to help those that are less technical understand some simple concepts to get you started on your own journey of learning, or just to help you feel better when talking with someone about the topic :->

If you are interested in incremental, multi-team delivery, you will find these concepts important to understand.  

There are other concepts and the following is definitely an over-simplification, but this should get you started.  My goal is not to make you an expert, but to get you ready to have some conversations about this in your environment.






  • Public interface
  • Keep the implementation private
  • Don't make something public unless you really have to

For the purpose of this post, consider an Interface to be .... 

The means by which you interact with something to send and/or receive information or change of state.

An API is an Application Program Interface.  To extend the definition, consider an API to be...  

The means by which you interact with a Program or System to send and/or receive information or change of state.


For the technical people out there.. Yes, I know this is an oversimplification.




The Automobile gas pedal.

Public Interface - To interface is simple. One moves the pedal forward to increase the flow of gas to the engine.  Decrease the distance forward and the opposite happens.

The results are known to us either a - vehicle going faster, or b - engine revving faster, c - vehicle going slower, d - engine revving slower.

It is important to understand, there is no interface through the gas pedal for speed, torque or anything else.  Those are other systems that have their own interfaces.

Keep the implementation private - Imaging for yourself that to know how to push the gas, you needed to know

- How many cyclinders the vehicle has
- Is it gas or electric
- What size engine
- What type of engine computer
- etc.

If you needed these to interact with the gas pedal, it would certainly be more difficult to drive.

Imagine that before you pressed the gas pedal, you had to calculate the force necessary based on engine size, the angle of the hill you were on, figure out which cylinders to turn on in what order to be able to press the gas.

Don't make something public unless you really have to.

These details are referred to as Implementation details. A good interface keeps this information private (not visible to the person or system using it).   

By using this approach, you could for instance, change your engine from one type to another, or get an updated engine computer.  You might have different results as you press the gas pedal with evidence from other interfaces (ie: speedometer), but you didn't need to change anything in the current interface or learn anything new to use it.

Don't make something public unless you really have to - This concept is extremely important for those that are growing systems incrementally or organically. Think about it this way... 

Once you make an interface public in a complex system, it may be almost impossible to remove that interface from future iterations.

This situation will be compounded in systems that take years or generations to change or where interaction from other systems may rely on these interfaces long into the future.

In the example of our car, let's assume that someone said "You know, we could make the gas pedal really awesome if we allowed the operator to press on the right or left of the pedal by angling their feet to give a different acceleration potential".  

This might make sense in your environment (let's say you have a specialised product for race car drivers and that would improve their lives significantly!). Innovation is cool that way.

My goal here is to have you consider this; Would a different interface be appropriate ( a hand control for instance). Could we have a separate control for Normal Acceleration or Faster Acceleration (think about this.. some vehicles have a Sport Mode).

There are two potential considerations should you decide to add this really cool interface Publicly (in your current application).

In a distributed application or system, something or someone will use a public interface! This means that you may need to have that ability in your system forever. You may never take it away (well, not easily at least).

If you realize later that you should have made a different interface (Sport Mode), it will be too late. You have made the interface public.  


  • People may have already created driving schools for the new accelerator system
  • Someone may have created a special shoe to use it
  • Control systems may have been built to control the pedal from a manual control that converts it to suitable pressures on the gas pedal 
  • And so on.
To complete the analogy to software or systems development, now anytime someone wants to use the interface they have to spend time and effort learning and understanding acceleration concepts when all they wanted to do was press the gas pedal.

I hope this helped the non-technical person to be able to have some knowledgeable conversations about Application Program Interfaces or APIs.

There are Interfaces at different levels in your vehicle. Examples include....

- Engine computer
- Fuel pump
- Light switch controller
- Engine itself (think electric)
- Speedometer
- Tachometer

Technically oriented people will be using this term often in the future. APIs and discussions will become more prevalent as technology becomes more pervasive. The Internet of Things (IoT) is mostly about APIs for starters.

Hopefully, I have given you a bit of a non-technical description that helps you out to start your learning journey about this topic.

Actually, even communication and conversation is a form of Interface between humans. That's a topic for another time :->

Enjoy.

Tuesday, December 13, 2016


A simple thought today....

One of the smartest people I ever met showed me that overcoming "fear of loss" would improve my life. 

He was right. 

I love change! :-)




This post is somewhat related to this one.


Sunday, December 11, 2016

Consider the principles of the Agile Manifesto carefully. A thinking tool.


How to read this post: 

Read the word or grouping of words.

When you see the "........." Stop, pause and think about the word or phrase that just passed. 

Imagine for yourself what it would take to have a Culture  that supports this one thought or idea. 

Try not to rush the evaluation of each phrase as it relates to your environment.

Then, move to the next word or group of words. 

At the end of the exercise, you will find the specific agile principle considered.

You could repeat this thought process for any of the principles of the Agile Manifesto.



I will use one of the principles as an example...




Build projects around   


.............


motivated individuals


.............

Give them the environment   


.............


Give

.............


support they need


.............


they need



.............


trust them


.............


trust them to get the job done


.............


trust them


.............


job done


.............


done


.............


"Build projects around motivated individuals. 
Give them the environment and support they need, 
and trust them to get the job done."

Imagine for yourself that your organizational culture has changed to the point where these principles are embraced and part of everyday life.

I know from experience that an in-depth discussion on any of these words or phrases and their meanings and implication could be enlightening for many.

The rest is up to you.




References:

Agile Manifesto - http://agilemanifesto.org/

Agile Maniesto Principles - http://agilemanifesto.org/principles.html







Tuesday, December 6, 2016

Techpost - The Test-Maintain Loop saved me during a Jenkins Server infrastructure upgrade

Technical Post (just a warning for the non-technical follower).

As some of you know, earlier this year I created a first version of an Ansible Playbook Test framework and put it on Ansible Galaxy.  


Yesterday, this tool saved me from creating a disaster on my Production Jenkins CI instance on AWS (Amazon Web Services). 

I'd like to share that story as an example for others of what is possible if you include testing as part of your Infrastructure as Code strategy.

How I was saved yesterday:

(some details removed for security and simplicity reasons).

Execute the Playbook...
ansible-playbook -i Inventory/CASPAR/staging/ CASPAR_setup_jenkinsservers.yml -u ubuntu --ask-vault-pass --ask-sudo-pass
The result is a new AWS instance server with Java 8 loaded,  appropriate users and groups setup and a base Jenkins Image loaded.  The key here is SETUP (the minimum needed to get a server into MAINTAIN status. The server is in "Staging".

Then, the repetitive playbook is executed (this one runs on a regular basis to keep servers up-to-date in all environments. In my case, I execute for Dev/Staging and Production ( the same playbook is used and applied to all 3 environments ).

ansible-playbook -i Inventory/CASPAR/staging/ CASPAR_maintain_jenkinsservers.yml -u ubuntu --ask-vault-pass --ask-sudo-pass

Note: The only difference in the name is the word "maintain". 

Note: To run the same playbook in Prod or Dev, I simply run the same playbook against /CASPAR/prod/  ( Ansible's dynamic Inventory auto-finds the appropriate machines based on tags )

Then, while the machine is still in Staging, the following test playbook is executed....
ansible-playbook -i Inventory/CASPAR/staging/ CASPAR_test_jenkinsservers.yml -u ubuntu --ask-vault-pass --ask-sudo-pass

The "test" playbook executes all predefined tests to ensure the server is in good shape. 
If there are no errors, all that needs to happen is for the machine to be re-tagged in AWS from Staging to Prod and then the next time the maintain playbook is executed, it will have any appropriate changes using the SAME playbook as before (different Gateway addresses, different database connection string, etc).

Then of course, the TEST playbook is executed again (one final test).. 

The Test playbook can now also serve as a Governance check playbook as well and could be executed by the same team or externally where needed. It provides a means for safer, more comfortable changes, while also providing a built-in governance component if needed.

Yesterday, when I ran the tests in Staging I received an error about missing packages.


ansible-playbook -i Inventory/CASPAR/staging/ CASPAR_test_jenkinsservers.yml -u ubuntu --ask-vault-pass --ask-sudo-pass > test.log
grep "TEST_PASSED" test.log
grep "TEST_FAILED" test.log

I received the message...

 "msg": "TEST_FAILED: package xxxxxx expected present "

(xxxxxx is a hidden package only for this post for security reasons).

If I had converted the host to Production, it would have caused big problems in my production environment.

After doing some research, If found that I had previously requested a newer version of an AMI ( an Amazon Machine Image ). 

Although the entire "setup" and "maintain" playbooks ran flawlessly (with no errors), what I did not know was the newer AMI was missing a critical Operating System package that my environment needed.

I modified the "maintain" playbook to include the missing package, re-ran the "maintain" playbook and then re-ran the Test Playbook.  Everything passed. Now, I know the Staging and Prod machines will always be up-to-date with this package when the "maintain" playbook runs it's continuous loop.

The new Jenkins Server was tagged as "Prod" and then the previous server deleted from AWS. The transition was painless.

By taking this approach and adding new checks to my server first as they become evident, I ensure that I will  not deploy something to production that has not already been determined to be a potential problem. 

I will no longer have this issue or one related to missing this package again.  If an image contains the missing image, no problem.. It will simply pass. Ansible does not re-install packages if they already exist (unless "latest" is specified in the version").



Brief History of the Test/Maintain/Govern Loop

The purpose of creating the Test/Maintain/Govern Loop for Playbooks was to show a Test-First approach to infrastructure delivery to make the transition to Infrastructure as code easier to get accustomed to.

The approach uses knowledge taken from years of insight from the software development world in delivery of complicated environments and applies it to the Infrastructure as Code domain.  


Technical Notes:

Jenkins CI server running in Production on AWS.  

Ansible Playbook uses to Setup/Maintain and Test server(s) in both Staging, and Production. ((how my environment works for build servers.. TODAY).

In AWS, tags are used to determine if a machine is "in production" or "in staging". They are both live in AWS in the same VPC (A VPC is like a private IP range within AWS for my hosts to reach each other).

Playbooks are formatted into YAML (a markup format) to have Dev/Staging/Prod in the same playbook.

A unique matching approach allows the same Playbook to run many times in Dev and Staging. This helps to ensure that when the Playbook runs on the Production machine, it has already executed many times already (and confirmed correct).

An often missing catch with playbooks is that "If" statements can be used to determine of parts of playbooks are executed.  A playbook command can be set to only run a certain instruction only IF a certain environment exists (an example).

When I want to upgrade my Jenkins server or reconfigure a new one, I take an approach of.. "Build a new one, run the setup/maintain and test on it, and IF everything is OK, move it into the Production Tag and then disable the older server. This allows me to ensure all is well before activating a new production change.  

Think of the saying ....  

"All Servers Are Temporary"



If you feel that your organization could benefit from learning about a Test First approach to Infrastructure, please feel free to reach out to me.  I provide 1/2 day or full day sessions in the Toronto area or full-day sessions plus expenses anywhere else worldwide. 

A link to the original presentation be found here..... 



If you are so inclined, here's a link to the root repository... 

A sample "test" (also used for governance) playbook is located here...    
https://github.com/MikeCaspar/playbook_test_framework/blob/master/sample/applicationX_proxy_servers_test.yml


Monday, November 28, 2016

Change can be fun or exciting


Over the last few weeks, I have seen a repeating theme in my social media feeds.

That theme... "Change is Hard" 

In my lifetime, I have been part of change, both positive and negative. 

Some of that change was imposed on me from above, and some from market forces. In some cases, personal interactions create change. My reaction to these different situations is very different for sure.

If you are a person who helps others to embrace or live through change (whatever your interpretation of change is)....

... consider the damage you are causing by inspiring fear where it simply may not be appropriate or necessary.


I can say from both personal and professional experience...


Change does not have to be hard.

It can be fun or exciting!


Please stop giving the impression that hard change is mandatory.








Sunday, November 13, 2016

Follow Rachel Perry as she learns to use Scrum to deliver a product to improve her community


Today I want to share an inspiring story about a person who is learning to apply agile values and principles through experimentation in both personal and business aspects of her life.

My first sign of this was from a post Rachel Perry made about trying to deliver a newsletter for the company she works for with her team. 

She took her understanding of Scrum to experiment with an approach to delivering a weekly newsletter using "Sprints" and a focus on learning. She blogs about it here at agileadvice.com

For those of you that are interested in using Scrum in non-IT environments, it's worth following some of Rachel's learning as she progresses in her journey.

She is also developing social service product
to help communities. Read about it here.


If you would like to learn more, I'm sure she'd appreciate you reaching out to her to learn more (or help in your community).


Good luck Rachel as you progress on your journey.



Wednesday, November 2, 2016

An agile approach helps educators see better financial results to support their students


Someone pointed out today that forgot to post a reference to an article for my usual blog followers... 


Image (c) Blueprint Education, 2016
These two excerpt are from an article published on the Scrum Alliance Member articles section on Aug 24, 2016. It talks a bit about how a new approach can help the business side of charter schools. 
"Central to this turnaround effort was the creation of an Agile culture."

"The discipline of focus produced great results. Relationships also grew deeper from working together through the discouraging times. The result is that the team has developed new competencies, along with an increased confidence." 



If you have an interest in some other posts from this blog about this topic, here are two searches that will give you some more reading material.

link - Agile in Education