INTRODUCTION

Big data is a term used to describe large, complex data sets that are difficult to process using traditional data processing methods. Big data is a rapidly growing field, and new applications for big data are being developed all the time. As big data becomes more widely used, it is likely to have a significant impact on businesses, governments, and society as a whole. It is characterised by its volume, velocity, variety, and veracity.

1. Volume refers to the sheer size of big data sets. Big data sets can be terabytes, petabytes, or even exabytes in size.

2. Velocity refers to the speed at which big data is generated and collected. Big data sets can be generated in real time, or they can be collected over a period of time.

3. Variety refers to the different types of data that can be included in big data sets. Big data sets can include structured data, unstructured data, and semi-structured data.

4. Veracity refers to the accuracy and reliability of big data sets. Big data sets can be noisy or incomplete, which can make it difficult to draw accurate conclusions from them.

Big data is often used to analyse trends, make predictions, and improve decision-making. Some examples of big data applications include:

1. Fraud detection - Big data can be used to identify patterns of fraudulent activity, such as credit card fraud or insurance fraud.

2. Customer segmentation - Big data can be used to segment customers into groups based on their demographics, interests, and purchase history. This information can then be used to target customers with specific marketing campaigns.

3. Product development - Big data can be used to collect feedback from customers and identify trends in product usage. This information can then be used to improve existing products or develop new products.

EXAMPLES OF BIG DATA

1. Social media data - This includes data from Twitter, Facebook, Instagram, LinkedIn, TikTok, Threads and other social media platforms.

2. Sensor data - This includes data from sensors that are used to monitor physical processes, such as temperature, humidity, and traffic flow.

3. Log data - This includes data from servers, applications, and devices.

4. Financial data - This includes data from stock markets, banks, and other financial institutions.

WHAT YOU WILL LEARN

1. 1. Hadoop and Apache Spark - two of the most popular big data frameworks. Hadoop is a distributed file system that is used to store and manage big data sets. Apache Spark is a unified analytics engine that can be used to process big data sets in real time.

   2. Business Intelligence mindset - the ability to think critically about data and use it to make informed decisions. It involves understanding the different types of data, how to collect and store data, and how to analyse data to identify trends and patterns.

   3. Data Optimisation - the skills that are used to improve the performance of big data applications such as data compression, data deduplication, and data caching.

   4. Emerging Technology Synthesis - the ability to understand and apply new technologies to big data problems. This includes technologies such as artificial intelligence, machine learning, and natural language processing.

   5. Solution Architecture - the process of designing and implementing big data solutions. This includes tasks such as identifying the business requirements, designing the data architecture, and selecting the right big data technologies.

   6. System Integration - the process of connecting different big data systems together. This includes tasks such as data mapping, data transformation, and security configuration.

WHO THIS BOOTCAMP IS FOR

This bootcamp focuses towards engineers from various fields and are:

1. People who are new to big data - If you are new to big data, this bootcamp will give you a comprehensive introduction to the field. You will learn about the different big data technologies, how to use them to solve real-world problems, and how to develop a career in big data.

2. People who want to change careers - If you are looking to change careers into big data, this bootcamp can help you get the skills that you need. You will learn about the most in-demand big data skills, and you will have the opportunity to build a portfolio of projects that you can use to show potential employers.

3. People who want to advance their careers - If you are already working in big data, this bootcamp can help you advance your career. You will learn about the latest big data technologies, and you will have the opportunity to network with other professionals in the field.

WHAT YOU WILL NEED

To take this course, you will need a computer with a working internet connection... and commitment to invest your time in up skilling your self in this new and exciting technologies.

HOW THIS COURSE WILL BENEFIT YOU

This bootcamp focuses towards engineers from various fields and are:

1. Learning the fundamentals of big data such as data collection, storage, processing, and analysis. This will give you a solid foundation in big data and prepare you for more advanced topics.

2. Learning the latest big data technologies such as Hadoop, Spark, or Hive. This will give you the skills you need to work with big data in the real world.

3. Developing your big data skills through hands-on exercises and projects. This will give you the experience you need to succeed in a big data career.

4. This masterclass can give you a competitive edge in the job market. As the demand for big data professionals grows, employers will be looking for candidates with the skills and experience.

JOB OPPORTUNITIES 

1. Data Scientist - Data scientists are responsible for collecting, cleaning, and analysing large datasets. They use their skills to extract insights that can be used to improve business decisions.

2. Big Data Engineer - Big data engineers are responsible for building and maintaining big data infrastructure. They work with Hadoop, Spark, and other big data technologies to ensure that data is stored and processed efficiently.

3. Business Intelligence Analyst - Business intelligence analysts use data to help businesses make better decisions. They work with data visualisation tools to create reports and dashboards that help businesses understand their data.

4. Data Architect - Data architects design and implement big data solutions. They work with stakeholders to understand the business needs and then design solutions that meet those needs.

5. Data Analyst - Data analysts use data to solve business problems. They work with data to identify trends, patterns, and anomalies. They then use this information to help businesses make better decisions.

GENERAL COURSE GUIDE

The bootcamp will be taught by experienced engineers who will help students learn the skills they need to be successful in the technology/engineering industry. It is divided into 6 sections which are outlined below. Breakdown schedule of each section:

1. FUNDAMENTALS OF HADOOP & APACHE SPARK

Course Objective:

The objective of this course is to provide students with the knowledge and skills they need to use Hadoop and Apache Spark to process and analyse large datasets. By the end of the course, students will be able to:

1. Understand the principles of big data processing

2. Install and configure Hadoop and Apache Spark

3. Write MapReduce and Spark applications

4. Use Hadoop and Apache Spark to process and analyse large datasets

Learning Outcomes:

By the end of this course, students will be able to:

1. Understand the concepts and fundamentals of cyber-security and its significance in the digital age.

2. Identify the various forms of cybercrime and their implications for individuals, organisations, and society.

3. Evaluate the key principles and strategies involved in cyber-security management.

4. Comprehend the nature and impact of cyber terrorism, including internet radicalisation and terrorist use of the internet.

5. Analyse the cyber-terrorism framework and its implications for national and international security.

6. Examine real-world case studies to understand the practical application of cyber-security principles.

Prerequisites:

1. Basic understanding of computer science

2. Basic understanding of programming

HADOOP & APACHE SPARK OUTLINE:

The course will be divided into the following modules:

Introduction to Big Data

1. What is big data?

2. The challenges of big data processing

3. The benefits of big data processing

Hadoop

1. Introduction to Hadoop

2. The Hadoop Distributed File System (HDFS)

3. MapReduce

4. YARN

Apache Spark

1. Introduction to Spark

2. Spark SQL

3. Spark Streaming

4. Spark MLlib

Programming with Hadoop

1. Pig

2. Hive

3. HBase

Hands-on Hadoop

1. Installing Hadoop

2. Creating a Hadoop cluster

3. Running MapReduce jobs

4. Using Pig

5. Using Hive

6. Using HBase

Spark vs. Hadoop

1. The differences between Spark and Hadoop

2. When to use Spark

3. When to use Hadoop

Spark Streaming

1. Introduction to Spark Streaming

2. Working with live data

3. Building streaming applications

Spark MLlib

1. Introduction to Spark MLlib

2. Machine learning algorithms

3. Building machine learning models

Hands-on Spark

1. Installing Spark

2. Creating a Spark cluster

3. Running Spark jobs

4. Using Spark Streaming

5. Using Spark MLlib

Hadoop Security

1. Introduction to Hadoop security

2. Securing Hadoop clusters

3. Best practices for Hadoop security

Hadoop Administration

1. Introduction to Hadoop administration

2. Managing Hadoop clusters

3. Troubleshooting Hadoop problems

Hadoop Use Cases

1. Real-world use cases for Hadoop

2. How Hadoop is being used to solve big data problems

Hadoop in the Cloud

1. Running Hadoop in the cloud

2. Benefits of running Hadoop in the cloud

3. Challenges of running Hadoop in the cloud

Conclusion

1. Summary of the course

2. Next steps for learning more about Hadoop and Spark

Assessment:

The course will be assessed through a combination of quizzes and exercises.

2. DECISION & BUSINESS INTELLIGENCE

Course Objective:

The objective of this course is to provide students with the knowledge and skills they need to develop a business intelligence mindset. By the end of the course, students will be able to:

1. Understand the importance of business intelligence in today's business world

2. Identify the different types of business intelligence

3. Apply business intelligence techniques to solve business problems

4. Communicate the results of business intelligence analysis

Learning Outcomes:

By the end of this course, students will be able to:

1. Define business intelligence and explain its importance in today's business world

2. Identify the different types of business intelligence

3. Apply business intelligence techniques to solve business problems

4. Communicate the results of business intelligence analysis

5. Develop a business intelligence mindset

Prerequisites:

1. Some basic understanding of data analysis

2. Some basic understanding of statistics

DECISION & BUSINESS INTELLIGENCE OUTLINE:

The course will be divided into the following modules:

Introduction to Business Intelligence

1. What is business intelligence?

2. Why is business intelligence important?

3. The different types of business intelligence

Data Analysis for Business Intelligence

1. Data mining

2. Statistical analysis

3. Predictive analytics

Business Intelligence Tools

1. Microsoft Excel

2. Tableau

Applying Business Intelligence to Solve Business Problems

1. Customer churn analysis

2. Sales forecasting

3. Inventory optimisation

Communicating the Results of Business Intelligence Analysis

1. Creating reports

2. Presenting data

Developing a Business Intelligence Mindset

1. The importance of asking questions

2. The importance of visualising data

3. The importance of storytelling

Assessment:

The course will be assessed through a combination of quizzes and exercises.

3. DATA OPTIMISATION 

Course Objective:

The objective of this course is to provide students with the knowledge and skills they need to optimise data for better decision-making. By the end of the course, students will be able to:

1. Understand the importance of data optimisation

2. Identify the different techniques for data optimisation

3. Apply data optimisation techniques to real-world datasets

4. Communicate the results of data optimisation analysis

Learning Outcomes:

By the end of this course, students will be able to:

1. Define data optimisation and explain its importance

2. Identify the different techniques for data optimisation

3. Apply data optimisation techniques to real-world datasets

4. Communicate the results of data optimisation analysis

5. Develop a data optimisation mindset

Prerequisites:

Have completed the above sections.

DATA OPTIMISATION OUTLINE:

The course will be divided into the following modules:

Introduction to Data Optimisation

1. What is data optimisation?

2. Why is data optimisation important?

3. The different types of data optimisation

Data Cleaning and Preparation

1. Data cleaning

2. Data preparation

Data Compression

1. Data compression techniques

2. The benefits of data compression

Data Sampling

1. Data sampling techniques

2. The benefits of data sampling

Data Encryption

1. Data encryption techniques

2. The benefits of data encryption

Data Visualisation

1. Data visualisation techniques

2. The benefits of data visualisation

Communicating the Results of Data Optimisation

1. Creating reports

2. Presenting data

Developing a Data Optimisation Mindset

1. The importance of asking questions

2. The importance of visualising data

3. The importance of storytelling

Assessment:

The course will be assessed through a combination of quizzes and exercises.

4. UNDERSTANDING NEW TECHNOLOGIES

Course Objective:

The objective of this course is to provide students with the knowledge and skills they need to understand and synthesise emerging technologies. By the end of the course, students will be able to:

1. Identify and understand the key concepts and technologies that are emerging

2. Identify the potential impact of emerging technologies on society and business

3. Develop skills in synthesising emerging technologies to solve real-world problems

Learning Outcomes:

By the end of this course, students will be able to:

1. Define emerging technologies and explain their importance

2. Identify the key concepts and technologies that are emerging

3. Analyse the potential impact of emerging technologies on society and business

4. Develop skills in synthesising emerging technologies to solve real-world problems

5. Develop a critical thinking mindset about emerging technologies

Prerequisites:

Have completed the previous sections.

UNDERSTANDING NEW TECHOLOGIES OUTLINE:

The course will be divided into the following modules:

Introduction to Emerging Technologies

1. What are emerging technologies?

2. Why are emerging technologies important?

3. The different types of emerging technologies

The Future of Technology

1. The future of artificial intelligence

2. The future of robotics

3. The future of quantum computing

The Impact of Emerging Technologies on Society

1. The impact of artificial intelligence on society

2. The impact of robotics on society

3. The impact of quantum computing on society

The Impact of Emerging Technologies on Business

1. The impact of artificial intelligence on business

2. The impact of robotics on business

3. The impact of quantum computing on business

Synthesising Emerging Technologies

1. How to synthesise emerging technologies

2. Case studies of emerging technologies

Developing a Critical Thinking Mindset

1. The importance of critical thinking

2. How to develop critical thinking skills

Conclusion

1. Summary of the course

2. Next steps for learning more about emerging technologies

Assessment:

The course will be assessed through a combination of quizzes and exercises.

5. SOLUTION ARCHITECTURE

Course Objective:

The objective of this course is to provide students with the knowledge and skills they need to design and implement IT solutions. By the end of the course, students will be able to:

1. Understand the principles of solution architecture

2. Design and implement IT solutions

3. Evaluate and select IT solutions

4. Communicate the benefits of IT solutions

Learning Outcomes:

By the end of this course, students will be able to:

1. Define solution architecture and explain its importance

2. Identify the different components of a solution architecture

3. Design and implement IT solutions that meet the needs of the business

4. Evaluate and select IT solutions that are cost-effective and scalable

5. Communicate the benefits of IT solutions to stakeholders

Prerequisites:

Have completed the previous sections.

SOLUTION ARCHITECTURE OUTLINE:

The course will be divided into the following modules:

Introduction to Solution Architecture

1. What is solution architecture?

2. Why is solution architecture important?

3. The different types of solution architectures

The Solution Architecture Process

1. The steps in the solution architecture process

2. The different roles involved in the solution architecture process

Designing IT Solutions

1. The different elements of an IT solution

2. The different design approaches

Implementing IT Solutions

1. The different implementation approaches

2. The different tools and technologies used for implementation

Evaluating and Selecting IT Solutions

1. The different evaluation criteria

2. The different selection methods

Communicating the Benefits of IT Solutions

1. The importance of communicating the benefits of IT solutions

2. The different communication methods

Conclusion

1. Summary of the course

2. Next steps for learning more about solution architecture

Assessment:

The course will be assessed through a combination of quizzes and exercises.

6. SYSTEM DESIGN & INTEGRATION

Course Objective:

The objective of this course is to provide students with the knowledge and skills they need to integrate different IT systems. By the end of the course, students will be able to:

1. Understand the principles of system integration

2. Identify the different challenges of system integration

3. Design and implement system integration solutions

4. Manage system integration projects

Learning Outcomes:

By the end of this course, students will be able to:

1. Define system integration and explain its importance

2. Identify the different challenges of system integration

3. Design and implement system integration solutions that meet the needs of the business

4. Manage system integration projects effectively

5. Communicate the benefits of system integration to stakeholders

Prerequisites:

Have completed the previous sections.

SYSTEM DESIGN & INTEGRATION OUTLINE:

The course will be divided into the following modules:

Introduction to System Integration

1. What is system integration?

2. Why is system integration important?

3. The different types of system integration

The System Integration Process

1. The steps in the system integration process

2. The different roles involved in the system integration process

Designing System Integration Solutions

1. The different elements of a system integration solution

2. The different design approaches

Implementing System Integration Solutions

1. The different implementation approaches

2. The different tools and technologies used for implementation

Managing System Integration Projects

1. The different project management methodologies

2. The different tools and techniques used for project management

Communicating the Benefits of System Integration

1. The importance of communicating the benefits of system integration

2. The different communication methods

Conclusion

1. Summary of the course

2. Next steps for learning more about system integration

Assessment:

The course will be assessed through a combination of quizzes and exercises.

7. HACKATHON

DURATION: HALF DAY | 3.5 HOURS

Hackathon Objective:

The objective of this hackathon is to leverage the skills and knowledge gained from the bootcamp courses to develop innovative solutions for creating smart cities. Participants will work in teams to develop data-driven applications that integrate various technologies, optimise data processing, and provide intelligent insights for urban planning and management.

Learning Outcomes:

By the end of the hackathon, students will be able to:

1. Define system integration and explain its importance

2. Identify the different challenges of system integration

3. Design and implement system integration solutions that meet the needs of the business

4. Manage system integration projects effectively

5. Communicate the benefits of system integration to stakeholders

Prerequisites:

Have completed the previous sections.

HACKATHON OUTLINE:

Hadoop and Apache Spark

Theme - Big Data Processing and Analytics for Smart Cities

Description - Participants will use Hadoop and Apache Spark to process and analyze large volumes of data collected from various sources within a city. This includes sensor data, social media feeds, transportation logs, and more.

Business Intelligence Mindset

Theme - Decision Support Systems for Urban Planning

Description - Participants will develop business intelligence solutions that enable data-driven decision-making for urban planners. They will create interactive dashboards, visualisations, and reports to provide insights on key metrics and trends related to city infrastructure, resource allocation, and citizen services.

Data Optimisation Skills

Theme - Data Cleansing and Integration for Smart Cities

Description - Participants will focus on optimising data quality and integration techniques. They will develop algorithms and workflows to clean and transform heterogeneous data sources into a unified format, ensuring accuracy and consistency for subsequent analysis and application development.

Emerging Technology Synthesis

Theme - Innovations for Sustainable Cities

Description - Participants will explore emerging technologies such as Internet of Things (IoT), artificial intelligence (AI), and machine learning (ML) to develop innovative solutions for energy optimisation, waste management, traffic control, and other sustainability challenges faced by cities.

Solution Architecture

Theme - Scalable and Secure Smart City Infrastructure

Description - Participants will design scalable and secure architectures for deploying smart city applications. They will consider factors such as data privacy, real-time processing, distributed systems, and integration with existing infrastructure, ensuring a robust foundation for their solutions.

System Integration

Theme - Seamless Integration of Smart City Components

Description - Participants will focus on integrating various systems and technologies required for a smart city ecosystem. This includes integrating data from different sources, coordinating communication between devices and applications, and ensuring interoperability among various components.

Assessment:

The course will be assessed through a combination of quizzes and exercises.

JUDGING CRITERIA

1. Innovation and Creativity - How unique and creative is the solution?

2. Impact and Relevance - Does the solution address a significant challenge faced by smart cities?

3. Technical Implementation - How well is the solution implemented from a technical perspective?

4. User Experience - Is the solution intuitive, user-friendly, and accessible?

5. Scalability and Sustainability - Can the solution scale to accommodate a large user base? Is it economically sustainable?

YOUR TRAINERS

Dr Khairul Anuar Abd Wahid -  is a Senior Trainer at Marc & Zed. He has over 15 years of experience in the IT industry, and has worked as a trainer, lecturer, and consultant for software development, data science, machine learning, artificial intelligence, and cloud computing.

He has taught in Singapore, Malaysia, and the United States. He started his teaching career in 2007 as a lecturer for software engineering at the National University of Malaysia. In 2012, he joined a Silicon Valley startup, where he worked on developing machine learning algorithms for fraud detection.

Since 2017, Dr. Khairul has conducted training and workshops in Python, R, Machine Learning, Artificial Intelligence, Cloud Computing, and Data Science. He is also a certified Data Scientist and Machine Learning Engineer. He is a highly experienced and qualified trainer, and has a wealth of knowledge and experience in the IT industry. He is passionate about teaching and helping others to learn, and is committed to providing high-quality training that meets the needs of his clients.

Dr. Khairul is a valuable asset to the Marc & Zed Training team, and his expertise in data science and machine learning is highly sought after by businesses in Singapore and Malaysia. He is a passionate educator who is committed to helping others learn and grow.

Djoshkun Diko - has been working as a developer, trainer, coach, and consultant in software engineering since 2008. His expertise includes FullStack, DevOps, Cloud Computing (Amazon Web Services & Google Cloud Platform), PHP, JavaScript, C++, Laravel, Docker, Kubernetes, Golang, VueJS, Python, Shell scripting, HTML5/CSS, MySQL, MariaDB, PostgreSQL, MSSQL Server, Cassandra, and MongoDB.

Throughout his career as a Software Architect/developer/trainer, he has been involved in designing and executing distributed system architecture principles and patterns for applied machine learning products. He has contributed to various projects involving technologies such as Laravel, Symfony, Prestashop, NodeJS, ExpressJS, VueJS, MySQL, MongoDB, PostgreSQL, Camunda Microservices architecture with gRP, GoLang/Python & Echo (Go framework), Flask & Panda libraries (Python Framework), Angular, Docker & Kubernetes, and JIRA & Confluence (Atlassian products).

During his freelance career, he has collaborated with several companies, developing web pages, web shops, and forums using platforms such as Joomla, Wordpress, vBulletin, MyBB, and HTML.

In 2017, he joined Marc & Zed SPACES in Kuala Lumpur as an Assistant Trainer. Although he left Marc & Zed in 2019, his interest in the training field brought him back in February 2020 as a Principal Trainer and Coach. In this role, he conducts hybrid trainings in Singapore, Germany, and Malaysia. He has also taken on web development projects for Marc & Zed, including developing their own website and creating a CMS website for propertysifu.com.my, and providing training for their staff. Currently, he is working on developing a website and providing training for another client of Marc & Zed, Cameron Adams UK Ltd., a real-estate agency.

Dr Harjinthar Singh -  is a Principal Trainer at Marc & Zed. He has over 25 years of experience in the IT industry, and has worked as a trainer, lecturer, and consultant for software development, product design, user interface, user experience, data analysis, RDBMS, video and image editing, and mobile development.

Dr. Singh has taught in Singapore, the United Kingdom, Malaysia, and Australia. He started his teaching career in 2001 as a lecturer for software engineering at London South Bank University. In 2012, he joined a Malaysian government agency, MIMOS Berhad. From 2016 to 2017, he taught Software & Mobile Development for undergraduates, staff re-training programmes, and post-graduates intending to pursue a career as programmers and developers.

Since 2017, Dr. Singh has conducted training and workshops in UI/UX, Interaction Design, Design Thinking, DevOps, MERN FullStack, Agile, JIRA, Git/GitLab, MySQL, MS SQL Server 2016, Infographics, Graphics/Video, and mobile/web development. He is also a certified Scrum Master and Product Owner.

Dr. Singh is a highly experienced and qualified trainer, and has a wealth of knowledge and experience in the IT industry. He is passionate about teaching and helping others to learn, and is committed to providing high-quality training that meets the needs of his clients.