Author: Abhishek Nag

CI/CD : Few things to remember
Today, CI/CD pipeline is hailed as one of the most important and beneficial practices for DevOps teams when it comes to delivering the code changes in a more reliable and quicker manner. Continuous Integration (CI) and Continuous Delivery (CD) refer to a set of operational principles, or basically a culture of following certain practices that enables development teams to deliver changes in the code more frequently and in a more reliable fashion.

CI/CD is an irreplaceable part of any development approach, irrelevant of what framework you are using, the tech stack or what your development practices are. The value addition of CI/CD is faster delivery, less deployment overhead to your development team. It enables you to deliver value in a quick, safe, and repeatable manner.

Purpose of CI/CD

Continuous Integration is a coding philosophy and group of practices. Their purpose is to enable the development teams to make small changes to the code and serves as a mechanism to integrate and validate those changes.

The purpose of CI is to automate your build pipeline. It is dependent on the tech stack that you are using. However, it enables your development teams to integrate your features, development changes in an automated way. It helps them to increase code quality, reduce errors and potential conflicts & deliver more value in less time.

Continuous Delivery comes into play where continuous integration exits the scene. It automates the process of application delivery to infrastructure environments. Basically, CD enables your operation team to integrate the end result over multiple environments. This way you can test your changes in multiple phases and make sure that we are getting value out of it. CD enables us to operationalize log aggregation, monitoring and other components. This is required for making sure smooth working of our end result.

CI Approach:
1. Improve the code quality: We can run the test cases and make sure that our code coverage is up to the par. We also can use static code analysis tools like sonarQube and codecy. This will be helpful to make sure that coding standards are maintained throughout the development process
2. Avoiding the code maintenance issues: One of the efficient ways to do this is to merge your code in the main/master branch frequently. This way, we can avoid last minute maintenance issues in code
3. Make sure that we enable automated testing and build deployment processes in the initial environments. This makes sure that we catch the issues early and improve the quality of code at faster pace.
CD Approach:
1. It’s best to follow a fail fast strategy in lower environments. This enables us to catch issues early on which increase stability in higher environments
2. Set up release & rollback strategy for each environment build. Things go wrong and we need to make sure that we have a plan in handy. In most of the cases, solution is move back to previous build artefacts .
CI/CD: Conclusion

So, we can see that CI/CD and its implementation requires the operations teams and the development teams to properly collaborate. This means they have to come together on technologies, priorities and practices. The different teams will have to build a consensus on which approach and technology they want to choose for their business. This is essential in order to make sure there is uniformity in following practices once CI/CD is in place.

If you would like to know more about CI/CD at EmbarkingonVoyage, please get in touch with us at info@embarkingonvoyage.com. We will be happy to help!
May 10, 2021
Transitioning from DevOps to NoOps

DevOps is the now widely used practice of having Developers and Ops work closely to enable faster, shorter releases to production. Instead of waiting longer times and preparing elaborate, risky releases, DevOps shortens the release process while still upholding quality and system integrity.

Then what is wrong with DevOps? Why would we need something like NoOps?

There is actually nothing wrong with DevOps, save a couple of operational challenges. NoOps is not built to tackle any limit but rather forge the path to higher organisational and operational maturity. Hence, it would make sense to think that unless you have a solid DevOps practice, there should be no strategising towards NoOps.

NoOps, exactly as the name says is very much the removal of Ops from the process. There are a few rationale behind wanting this.

1) Developers who monitor production environments are likely to catch issues much quicker than Ops who would rely on the customer to report.

2) It enables even faster releases and puts the onus of quality and environment protection on the developers and the tools in place for NoOps.

3) NoOps frees up the developers in DevOps to work on features or do something else.

4) Ops is difficult to scale. When you deal with more complex features and products, getting Ops to scale is a difficult process which needs a lot of adapting to the DevOps mindset.

How does one transition from DevOps to NoOps?

1) Mirroring

This would seem quite obvious, but this is very crucial to start moving to NoOps. There must be a non-production environment (like a sandbox) which completely mirrors the production environment. No exceptions. This would ensure that the final regression and functional quality checks are going to hold good in production as well. Without this, no amount of automated testing or tooling will ensure that you won’t need ops to keep looking over their shoulders after every release.

2) Cultural shift

Getting to NoOps is no mean feat, and hence a lot of friction needs to be absorbed. There may be reports of “but this is not working”. For any such instance, diagnose – > revamp -> redeploy. The core issue (say a missed bug in production) can be caught the next time by improving the process or strengthening the net to “catch” better. Of course, a great vendor is imperative to help craft and re-craft these improvements. But the key thing is to keep plowing on. Remember, NoOps is 0 ops, not sometimes 1 or 2 ops.

3) Empower the developers

The developers will own the production environments’ issues in NoOps. It is important to keep their hassles to a minimum. This can be done using tools to analyse and report production problems. With the right tooling, the majority of issues can be preemptively reported. It does not stop there. Tools can also help take minimise impact by running some pre-configured scripts upon report of specific issues. With the right tooling and strategy, you will see a matrix of different problem scenarios and the processes and the tools that kick in for each of them.

4) Get everyone on the same page

Developers, testers, product owners, senior management- must understand and accept the NoOps implications. There will be a phase of “pain” when the NoOps practice kicks in, not to forget the apprehension. Hence, it is important to cover all concerns in separate workshops with the stakeholders. “But what about that weekly report that the Ops used to send?” “What will happen to customer requests and not bugs?” “What will happen if no developer is available?” Get all those pesky questions out (make sure your IT partner tags along for these discussions). Each questions’s answer (if needed- because we will also prioritise the concerns based on cost, impact, frequency) will help the process to become sharper for your business context.

All this sounds like a lot of work? Well it is! But the good news is that, NoOps practices are slowly taking shape in the industry. So, chances are that you would move very smoothly and none of the things that worry you would actually come true. If you don’t believe us, then why don’t you drop an email to info@embarkingonvoyage.com and we can discuss what your thoughts are and find the best NoOps strategy for you.

April 26, 2021
Artificial intelligence and the e-Commerce market

Today, with the advent of more and more technological advancement, almost all the e-Commerce players are leveraging the power of Artificial Intelligence (AI). This is mainly to enhance their customer experience and revolutionise the way common operations are carried out.

In this blog, we will look at some of the ways in which AI is changing the e-commerce marketplace scenario and helping such firms in their growth and scalability.

Optimization of warehouse operations

Bigger e-commerce players such as Amazon have huge fulfilment centres, spread globally. These firms are looking into millions of orders on a daily basis. Can you imagine the complexity of this process? How does one find where a particular product is? How to fulfil the order in the stipulated time? For this purpose, Amazon uses AI-powered robots that help in grouping, placing and arranging the packages to reduce the time taken to find and pack them. Well, this is just one example from the lot!

Chatbots

More and more businesses are adopting chatbots for their e-commerce stores. They are growing increasingly popular and all customers are falling in love with them! With fully-functional chatbots in place, customers can automatically find answers to their queries through text chats. In all of this, you won’t have to worry a bit! Chatbots are a wonderful product of artificial intelligence which facilitate businesses to offer round-the-clock customer support to their target audience. Another important factor is that they help to collect important data. Through chatbots, you can also track consumer behaviour and fuel your conversion rates.

Image recognition to find products

It is a quite common occurrence for all of us to like something and wanting to purchase it. however, we are not sure what it is called or where exactly to search for it. This is another way AI has been revolutionising the online marketplaces. So, the next time you want a particular item that you have fallen in love with, what to do? Take a picture and upload it and the apps powered for the same, will show you all related products. Having this feature integrated to your e-commerce site will give you the leverage of tapping a wide range of customers.

Customer Relationship Management

The whole world hails artificial intelligence for its predictive nature. Traditional CRM has been completely transformed by artificial intelligence. Gone are the days when one had to collect huge amounts of data and go through them individually to make conclusions and predictions. Today, we have AI and its power, to consume all the data that is coming over to you. And, process them, without any human intervention. AI processes the data and gives out important information to you. this includes who is more likely to purchase from you, what is their area of interest. It also tells you what is it that they will likely purchase and how you can engage with them to encourage conversions. WOW!

Quality advertising

Earlier, there used to be a lot of aggressive marketing and re-marketing along with increase ad exposure. With the advent of AI, the new marketing world has been focussing more on quality. As a result, it stresses on the importance of pushing more relevant ads to the right visitors at the correct time. This will help to connect better with your audience, give them what they want and encourage them to buy more from you.

Smart Technology

We have all heard of IoT or Internet of Things. Earlier internet meant only a group of computers and laptops connected together. Today, more and more devices and gadgets are coming to be connected to the internet. We have smart cars, smart watches, smart refrigerators and so on! So, if you are also looking to tap the advantage of AI to your e-commerce setup, you should really not delay it further.

Conclusion

From this above discussion, we can see that artificial intelligence is constantly helping e-commerce platforms to enhance the experience for the customers. With better recommendations, advertisements, smart technology and chatbots etc, AI has been changing the game with e-commerce platforms.

If you are also interested in using AI for your benefit, you are at the right place. At EOV, we have a team of experts who will help you in incorporating AI to your e-commerce platform. This will in turn help you give out better services to your customers. You can reach out to us at info@embarkingonvoyage.com to know how we can help you.

April 12, 2021
Data Extraction using Python Libraries
Introduction

Today, we are surrounded by data everywhere. Data has become easily accessible. So, the challenge that arises out of it is how to make the most of the available data! The first step towards using such vast amounts of data is finding the right data integration tool that could help you to study, analyse and manage dynamically different data from numerous sources. However, the bigger challenge before using the data is data extraction.

Therefore, now we are going to see in details what exactly this extraction of data is, what tools are available for the same and what role it plays in integrating data.

What is data extraction?

In simple words, it is the collection of different types of data from multiple sources, most of which are unorganised or purely unstructured.

Data extraction is mainly about consolidating, processing and refining the unstructured data and storing it on a centralised location for further transformation. You may store it on-site, or on cloud based platforms or a hybrid of both.

Data Extraction and ETL: How does the process work?

Let us have a brief look at the ETL process for a better understanding. With the help of ETL, the companies can collect data from different sources and store them on a centralised location and assimilate various and differing data into a common and understandable format. Basically, the ETL process involves:
1. Extraction: This process mainly deals with getting the data from various different sources. The extraction finds and locates relevant data and makes it suitable for further processing.
2. Transformation: After extraction is complete, it is now time for refining the data. During this process, the data is organised and cleansed. The main elements in this process include erasing the duplicate entries, removing the missing values etc. At the end of the transformation phase, what we are left with is reliable, and usable data.
3. Loading: Once the transformation of data is complete, the processed and high-quality data is loaded onto a centralised storage location for further use and analysis.
Many companies use data extraction for a number of reasons. It could be to streamline processes or support compliant efforts or so on.

Because now we are clear about what the process of data extraction is, let us have a look at what are the tools or methodologies available to extract the data.

Types of Data Extraction Tools

When it comes to extracting data, the two key decisions that data engineers have to take while designing the process are

What method to choose for extraction?

When it comes to selecting the extraction method, there are two options with the data engineers. They can go for either logical or physical modes of extraction. Under the logical extraction, there are further two ways – full extraction and incremental extraction.

Now, let us look at these extraction methods in brief.

Physical extraction

Sometimes, there could be certain limitation with the source systems. Say for example, if you are trying to extract data from an outdated data storage unit, you will not be able to do it using logical extraction and you are left with only the physical way to do it. There are two types of physical extraction

Online extraction – where data is directly transferred from the source to the data warehouse by directly connecting the extraction tools to the source system or the transitional system.

Offline extraction – where there is no direct extraction and the process has to be carried out outside the source unit. In this process, the data in question is already organised.

Logical Extraction

There are two kinds of logical extraction:

Full extraction: Under this process, all the data is extracted from the source system at one go directly. Any need for extra information, be it logical or technological, does not arise. For example, if you are trying to export a file on price change, the system will extract the entire financial records of the organisation.

Incremental extraction: This process deals with the incremental or delta changes in the data. The extraction tool recognises new or altered information based on date and time. If you are using this method, you need to add complex extraction logic to the source systems first.

What are the two libraries you would need to scrape website data on Python?

To extract data from web pages, some of the Popular Python Libraries to Perform Web Scraping include

lxml Library

It is another versatile Python library that deals with HTML and XML files. It is relatively fast and easy to use.

How to install it?

We can use the pip command to install lxml.
```
(base) D:\ProgramData>pip install lxml
Collecting lxml
   Downloading
https://files.pythonhosted.org/packages/b9/55/bcc78c70e8ba30f51b5495eb0e
3e949aa06e4a2de55b3de53dc9fa9653fa/lxml-4.2.5-cp36-cp36m-win_amd64.whl
(3.
6MB)
   100% |¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦¦| 3.6MB 64kB/s
Installing collected packages: lxml
Successfully installed lxml-4.2.5
```
Beautiful Soup Library for Web Scraping

Let’s consider the case where you are looking to collect al the hyperlinks from any web page. In such cases, we can use Beautiful Soup Python library. It is mainly used to pull data out of HTML and XML files. You can use it with requests because it can’t fetch a web page on its own and needs an input to process.

How to install it?

We use the pip command to install beautiulsoup.
```
(base) D:\ProgramData>pip install bs4
Collecting bs4
   Downloading
https://files.pythonhosted.org/packages/10/ed/7e8b97591f6f456174139ec089c769f89
a94a1a4025fe967691de971f314/bs4-0.0.1.tar.gz
Requirement already satisfied: beautifulsoup4 in d:\programdata\lib\sitepackages
(from bs4) (4.6.0)
Building wheels for collected packages: bs4
   Running setup.py bdist_wheel for bs4 ... done
   Stored in directory:
C:\Users\gaurav\AppData\Local\pip\Cache\wheels\a0\b0\b2\4f80b9456b87abedbc0bf2d
52235414c3467d8889be38dd472
Successfully built bs4
Installing collected packages: bs4
Successfully installed bs4-0.0.1
```
Extracting Data with EOV

EmbarkingOnVoyage has been a successfully leading the data extraction field, with an adept knowledge in multilingual text analytics. So, if you would like to know how we can help you in extraction of required data, please feel free to get in touch with us at info@embarkingonvoyage.com today!
March 1, 2021
Design Thinking in Product Development
Design is the basic or hygiene towards a successful product development. The concept between Product Design and Design Thinking is extremely thin, although both look identical. Design thinking is more from end-to-end product development. In Product Design, the designers focus on the problem statement, end goals and product users.

The Design Thinking approach fits very well in product designing. Product design focuses in creating the right applications, matched with the right technology and thereby, extending to user to establish right experience. The product customer value is concluded on the basis of the experience across the customer decision journey. Design thinking approach involves creative and systematic approach to problem solving, keeping customer first.

Let’s understand how Design thinking approach is followed in the Product Management. To start with, the Design Thinking methodology focuses on inspiration, purpose, iteration and lesser ambiguity as the development begins. In other words, Design thinking shows the point of intersection between purpose, feasibility and viability. The Design Thinking is somehow close to agile methodologies.

There are six ways through which Product Managers apply Design Thinking:
1. Being Creative while undergoing research – In design thinking, it’s always wise to be as creative as one can be. During research several ideas come as an option. However, creativity pulls the best out of research and most often delivers differentiated customer experience.
2. Define particular occurrence during product development – Clearly communicate the challenges, purpose and users. Every users and challenges will have distinct persona and so, the journey varies. In product development, it’s a good practice to have the problem statement defined at basic level.
3. Building prototypes – Prototyping is a quick and inexpensive way to see how the idea works, so business can go back to the users and get their feedback.
4. Testing of Prototypes – Testing or feedback gives information to both business and development team on the usability and experience. Mostly, from the users’ reactions, business discovers the problem statement which we started to addressed is not there and there’s a different problem.
5. Adapt design thinking tools – Adoption of design tools facilitate the Design Thinking innovative process. Since design thinking comprises a set five stages process: empathizing, defining, ideating, prototyping, and testing, selecting the right tools is absolutely the most important thing for effective decision making and constructive communication in a multidisciplinary team. Tools can be physical, such as a pen, paper, and whiteboard, or software applications having rich graphics that compliment the Design Thinking process. The tools can also be used to help teams in adopting a new perspective on design tasks, to visualize the system’s complexity and depending on the design stage reflect a convergent or divergent view of design.
6. Retrospective of the complete process – Design thinking focuses on the human-centered goals because it focuses on providing deep and meaningful engagement with the end users. There are some problems that are not solvable. You might not find a technology that’s going to solve a particular problem, but what you want to do is discover that quickly. Design thinking makes it possible!
  So, the design thinking methodology doesn’t necessarily generate better ideas than competing methodologies. It’s just that this methodology allows you to test your ideas quickly to see which ones hold promise.
February 1, 2021
Role of Digital Product Engineering in Business Digitization

Introduction

All product companies globally are moving towards digitization. The fundamental purpose for product companies to move into digital is to deliver innovative, customer focused solutions to meet the end business purpose. In order to be globally competitive, it is important for businesses to develop and deliver innovative, customer-focused products and services. What’s important here is that this needs to be done at ever shorter intervals. To be on the road of digitization, it’s important to start from basics like purpose, data sources, role of data analysis and intelligence, purpose of digital tool and tools being used at the moment and most important is cyber security.
It’s natural when we start moving towards being digital, it involves several investments and such investment does not guarantees quick turnaround but it gives direction as to how strategically organizations resources can be used. While a company starts moving towards being digital, according to the recent PWC report, with digitization kicking in enterprise efficiencies will certainly increase by 19% over next half a decade. This means on average the time market will also drop by close to 20% and increasing productivity of people by another 20%.

Some advantages of digital product engineering

The advantage which companies can expect while investment in digital product engineering:

The digital product engineering has certainly increased and strengthens the relationship with customer purposes. Customized offerings to help client in accelerating revenue making is the core in digital product engineering. The major challenge is to achieve the end result with available resources with no additional expenses.
In order to ensure successful digital strategy, it’s important to introduce the concept of customer focus right at day 1. Another study conducted by PWC finds out that an introduction of such concept will increase the share of personalized solution offerings in the next five years by over 24%.
The success in the digital journey is heavily dependent on the use of data and AI.

Usage of Data Analytics & Artificial Intelligence

Digital tools have gained significant importance in digital engineering. Such tools include use of Artificial Intelligence and data analytics. Almost 66% of the companies are adapting digital use tools for co-creating products and services. This includes both with internal and or with external partners. Almost 50% of companies use digital technologies for process simulation and the development of digital prototypes.

Role of cyber security

While we talk about digital, it’s extremely important to understand the role of cyber security. Almost 71% of the companies entered into digital engineering do not have a matured process to mitigate cyber threats in data driven development environment. This is according to a recent study published by PWC. The cost implications of cyber attacks take toll on both business financials and client trust. Thus, security must be considered throughout the product development lifecycle and the protection of all data systems. Therefore, security should be layered throughout the product development lifecycle, built from the scratch and not towards conclusion.

Steps to implement cyber security in product development engineering
1. Start from basics or idea stage and keep the process evolve
2. Quality is a journey and not end destination
3. Train employee on data security, cyber threats and business implications
4. Design Experience with future in anticipation, as product will evolve more dynamically than business
5. Product or Data security at every level of development / delivery

Conclusion

So, we can clearly see that digital product engineering is extremely crucial for companies that are planning to go digital. So, do not be left behind! Contact info@embarkingonvoyage.com to know how we can work together to make your digitization journey successful!

January 18, 2021