Snowflake's Evolution: A Look at the New Features in 2024

AWS Systems Manager seamlessly operates with both Windows and Linux OS and integrates with CloudWatch metrics, CloudWatch Dashboard, and AWS Config. Moreover, it enables the creation of resource groups spanning various AWS services, allowing for aggregated operational data viewing and facilitating monitoring, troubleshooting, and resource group-specific actions.

Common use cases and best practices for AWS Systems Manager capabilities are listed below:

Automation

Create self-service Automation runbooks for infrastructure.

Leverage AWS Systems Manager's Automation feature to simplify the process of generating Amazon Machine Images (AMIs) from the AWS Marketplace or custom AMIs. This can be achieved using public Systems Manager documents (SSM documents) or by creating custom workflows tailored to your specific needs.

Two recommended Automation runbooks for building and maintaining AMIs are AWS-UpdateLinuxAmi and AWS-UpdateWindowsAmi.

Inventory

Audit Application Configurations Over Time with Inventory

Use AWS Systems Manager's Inventory in conjunction with AWS Config to conduct comprehensive audits of your application configurations as they evolve over time.

Maintenance Windows

Scheduled Maintenance for Node Management

Define a routine schedule for executing potentially disruptive actions on your nodes, including activities like operating system (OS) patching, driver updates, or software installations.

For guidance on choosing between State Manager and Maintenance Windows, both key capabilities of AWS Systems Manager, refer to "Choosing between State Manager and Maintenance Windows."

Parameter Store

Centralised Management of Global Configuration Settings

Employ Parameter Store, a pivotal feature within AWS Systems Manager, to centrally manage and maintain global configuration settings.

Patch Manager

Efficient Patch Rollouts and Enhanced Fleet Compliance

Make use of Patch Manager, an integral component of AWS Systems Manager, to orchestrate the deployment of patches across your fleet of instances while also gaining enhanced visibility into fleet compliance.

To bolster your security posture, integrate Patch Manager with AWS Security Hub for real-time compliance alerts and monitoring of patching status.

Optimise Patch Compliance Scans

It is advisable to employ only one method at a time for scanning managed nodes for patch compliance to prevent accidental overwriting of compliance data.

Run Command

Effortless Instance Management without SSH Access

Harness the power of EC2 Run Command within AWS Systems Manager for managing instances at scale without the need for SSH access.

Secure and Controlled Command Execution

Audit all API calls related to Run Command by utilising AWS CloudTrail. To maintain data security, avoid including sensitive information in plaintext when sending commands via Run Command. Utilise SecureString parameters for encryption of sensitive data used in Systems Manager operations.

Staged Command Execution and Access Control

Employ the targets and rate control features in Run Command for executing staged command operations and ensure fine-grained access permissions through AWS Identity and Access Management (IAM) policies.

Session Manager

Comprehensive Session Activity Auditing

Audit session activity within your AWS account by making use of AWS CloudTrail.

Session Data Logging and Access Control

Log session data in your AWS account via Amazon CloudWatch Logs or Amazon S3, enabling tighter control over user session access to instances. Additionally, you can manage access to commands within a session and customise SSM-user account administrative permissions.

State Manager

Regular SSM Agent Updates and Tags for Node Management

Ensure that SSM Agent is updated at least once a month by utilising the pre-configured AWS-UpdateSSMAgent document. For Windows instances, upload PowerShell or DSC modules to Amazon S3 and use AWS-InstallPowerShellModule.

Employ tags to create application groups for your nodes and target nodes using the Targets parameter instead of specifying individual node IDs. Automate the remediation of findings generated by Amazon Inspector using Systems Manager.

Centralized Configuration Repository and Document Sharing

Utilize a centralised configuration repository for your SSM documents and share these documents across your organisation for streamlined management.

Managed Nodes

Time Accuracy for Optimal Operations

Accurate time references are essential for Systems Manager to perform operations effectively. Ensure that your node's date and time settings are correct, as discrepancies may lead to errors or incomplete functionality, and can result in nodes not being included in your managed nodes lists.

Case Study: A Global Cloud Solutions and Services Company Enhances Scalability and Efficiency with AWS Systems Manager

Client: A Global Cloud Solutions and Services Company

A technology services company that specialises in helping organisations across 120 countries adopt modern technologies and manage them efficiently. They focus on creating solutions for hybrid and multi-cloud environments.

Requirement: Finding Scalability on AWS Systems Manager

The client faced a significant challenge in managing multi-cloud environments at scale reliably and cost-effectively. Manually handling activities across hundreds of thousands of different compute instances was resource-intensive and delayed issue resolution. They needed a solution that could run both on-premises and on the cloud and wanted a single tool for managing their suite of solutions.>
Challenges

Manual management of hundreds of thousands of compute instances.

Delays in issue resolution due to human intervention.

Need for a solution that could operate on-premises and in the cloud.

Desire for a unified tool to manage their suite of solutions.

Process: Supporting Automation, Staff Productivity, and Transparency on AWS

The client began using AWS Systems Manager in 2015 for various products and extended its use to other cloud environments in 2019. Since 2019, the client has utilised AWS Systems Manager to power patching activities across all major cloud providers they support. They perform mass patching at scale, covering over 62,000 VMs across all their managed services. VM Management automates traditionally manual tasks like patching, agent distribution, server diagnostics, and issue remediation. It significantly reduces labour, costs, and errors associated with manual tasks, enhancing security and efficiency.

SmartTickets, a component in VM Management, handled thousands of incidents and automated responses using AWS Systems Manager, saving time and reducing costs for the company. They also used Amazon CloudWatch for monitoring and observability and automated runbooks for real-time monitoring and alerts.

AWS Systems Manager provides a single-pane view of environments, improving customer visibility and decision-making.

Result: Taking Automation to the Next Level on AWS

The client plans to develop custom runbooks with customers and further automate responses and resolutions using AWS Systems Manager. They have successfully solved industry challenges by saving time, cutting costs, and reducing complexity for both their customers and themselves.

Key Takeaway

The client leveraged AWS Systems Manager to streamline and automate their operations, resulting in improved efficiency, cost reduction, and enhanced customer satisfaction. With automation, they can swiftly respond to and resolve issues, meeting customer expectations effectively.

How Beinex Can Help You

Beinex is an AWS consulting partner, and we empower customers to host their BI solutions and much more on the cloud. Our cloud migration experts bring in best-in-class stability and reliability by understanding your business strategy and working closely with you to deploy AWS infrastructure as a service.

As a frequent user of Tableau Desktop and Server, check out my top picks from the list. • Improved dynamic parameter • Multiple marks layer support for maps • Block comments in calculations • Web authoring enhancements • Offline map support for Tableau Server • Tableau Server Management (TSM) improvements • Multiple key activation on Tableau Server prior to TSM initialization • Analytics Extension for Tableau Online • Merge duplicate external assets • Tableau Prep Builder in the browser Tableau Desktop Improved Dynamic Parameter The dynamic parameter was one of the features which I used extensively starting from 2020.1. The feature allows us to populate parameter values based on a specified field whenever the workbook is opened. We will be able to select only the field with matching data types. The feature update announced in Tableau 2020.4 gives me more privilege to customize the Dynamic Parameter. I can now set a parameter’s range to a field’s min and max value upon opening the workbook, or when the field’s data source is refreshed. So, whenever I open the workbook or the extract is refreshed, Tableau displays the unique values within the range specified, giving more control over the parameter values displayed. Multiple Marks Layer Support for Maps This is an exceptional feature to bring multiple spatial layers and context together to better understand and analyze geospatial data and map views. I will be able to include multiple marks layers from a data source to map visualizations and enhance the geospatial analysis. I can present more context in a single map view and perform further analysis with this feature. Block Comments in Calculations Block comments is a simple yet one of the most useful features for me, which Tableau has announced in 2020.4. I often used to add comments in complex calculations for easy understanding in future references. Earlier, only single-line comments were possible, limiting the description I could add. But from now on, I can add comments of any length to calculation windows with block comments by simply starting the comment with /* and ending with */. What makes it unique is that the new multi-line block comment feature is consistent with other popular programming languages. This feature is an example of Tableau’s on-going effort to provide its customers with an intuitive user experience. Tableau Server Web Authoring Enhancements Starting from Tableau 2020.4, it is possible to author dashboards from the browser just like how we design it in Tableau Desktop. With this web authoring enhancements, I can include Highlight actions, Format Mark Labels, apply filters to worksheets, create fixed sets, and even create extract in my workbooks from the browser itself. I no longer have to make changes from Tableau Desktop and publish it to the server. I can directly do it from the browser itself. Offline Map Support for Tableau Server Rendering dashboards with maps is now faster compared to previous versions. Now, I can create maps using the offline map style in web authoring, ensuring the performance of map views in Tableau Server. Offline map support is a great deal for organizations with strict internet access restrictions, assuring that map view access to all its users. Tableau Server Management (TSM) Improvements Tableau Server Administration activities like installation, upgrade and backup are now easy like never before. I can retry installation or upgrade from the last checkpoint in case of an unexpected issue or error during installation or upgrade, saving my efforts to obliterate tableau server. Backups can be performed twice as fast as previous versions, and I can monitor the progress with the new progress bar giving visibility into what step the backup is on and how much time is remaining. Backups can also be scheduled using TSM command starting from 2020.4 and that is awesome. I no longer need to prepare batch script and depend on the windows task scheduler to schedule the backups on regular intervals; instead, I can schedule it with just a single command. Multiple Key Activation on Tableau Server Prior to TSM Initialization During Tableau Server installation, it is now possible to activate multiple license keys prior to TSM initialization. I will be able to save a lot of time by eliminating the need to restart after the installation is completed to activate multiple licenses and experience a smoother installation. Analytics Extension for Tableau Online The power of the Analytics extension is now unlocked in Tableau Online too. The feature was already available in Tableau Server, and it helped us dynamically perform advanced analysis with models and functions in R, Python and other platforms. Analytics extension in Tableau Online significantly enhances the scope of using Advanced Analytics by the common users. Merge Duplicate External Assets Earlier, the Database or Table with similar names used to appear as multiple assets within Tableau Catalog. But, the new feature helps me to merge those multiple assets into a single one. I can manage assets easily and keep an organized view of External assets by merging the common ones. Tableau Prep Tableau Prep Builder in the Browser Prepare the data for visualizations from anywhere using a browser! Starting from 2020.4, Tableau is bringing the data prep process into one integrated platform on the web. Now I can easily prepare and manage prep flows from anywhere using a browser. Conclusion Tableau is progressively evolving as a single platform for data preparations, visualization, and collaboration with every update and version release. Author : Firdous Maqbool Images Courtesy : Tableau

AWS Storage Solutions

AWS offers a comprehensive array of storage solutions tailored to specific use cases and demands. These include Amazon S3, Amazon EBS, Amazon EFS, Amazon Glacier, AWS Storage Gateway, Amazon FSx, and the AWS Snow Family. Each service is designed to address storage needs, from simple data storage to high-performance file systems and large-scale data transfer.

Here are some of the key storage solutions provided by AWS: Amazon S3 (Simple Storage Service):

Object storage service for storing and retrieving large amounts of data.

Highly scalable, durable, and cost-effective.

Supports various storage classes for different use cases, including S3 Standard, S3 Intelligent-Tiering, S3 One Zone-IA, S3 Glacier, and more.

Suitable for data lakes, backups, static website hosting, and more.

Amazon EBS (Elastic Block Store):

Provides block-level storage volumes that can be attached to Amazon EC2 instances.

Supports different volume types, including General Purpose (SSD), Provisioned IOPS (SSD), Cold HDD, and Throughput Optimized HDD.

Used for running databases, applications, and workloads that require low-latency storage.

Amazon EFS (Elastic File System):

Manages file storage service that can be mounted to multiple EC2 instances.

Supports NFS (Network File System) protocol.

Ideal for shared file storage, content management, and data backup.

Amazon Glacier:

Low-cost archival storage service for data that is rarely accessed.

Suitable for long-term data retention and compliance requirements.

AWS Storage Gateway:

A hybrid cloud storage service that connects on-premises environments with cloud storage.

Supports file, volume, and tape gateways.

Amazon FSx:

Managed file storage for Windows (Amazon FSx for Windows File Server) and Lustre (Amazon FSx for Lustre) file systems.

AWS Snow Family:

A range of physical devices for transferring large amounts of data into and out of AWS when network transfer is impractical.

Latest Developments in AWS Storage Services

Over the past two quarters, AWS has introduced several exciting enhancements to its storage services, offering users greater flexibility and efficiency in managing their data. Let's delve into these notable updates:

AWS Backup's Local Time Zone Support

AWS Backup is a fully managed service that simplifies data protection across AWS services and hybrid workloads. The latest enhancement allows users to select their local time zone when creating or modifying backup plans. This eliminates the need for manual time zone conversions from UTC, making backup scheduling more user-friendly and convenient.

Amazon S3's Multivalue Answer for DNS Queries

Amazon S3 now offers Multivalue Answer (MVA) support for responding to DNS queries for S3 endpoints. With MVA, users can obtain up to eight S3 IP addresses per DNS query. This feature enables the automatic establishment of multiple concurrent connections to S3, potentially boosting throughput significantly. Moreover, MVA enhances retry operation efficiency by allowing applications to automatically attempt an alternative IP address without waiting for another DNS query. These improvements give users more robust and efficient access to Amazon S3 services.

General Availability of Mountpoint for Amazon S3

AWS has announced the general availability of Mountpoint for Amazon S3, an open-source file client that offers high-throughput access to Amazon Simple Storage Service (Amazon S3). This innovation reduces processing times and compute costs for data lake applications. Mountpoint for Amazon S3 acts as a file client that translates local file system API calls into S3 object API calls, including GET and PUT operations.

This feature is particularly valuable for workloads involving reading large datasets, ranging from terabytes to petabytes in size, and requiring the scalability and high throughput offered by Amazon S3. Use cases include machine learning training, reprocessing, autonomous vehicle data processing validation, and more. It supports sequential and random read operations on existing files and sequential write operations for creating new files. These capabilities enhance the efficiency of processing large-scale data stored in Amazon S3, making it a valuable tool for data-intensive applications.

Amazon S3 Glacier Flexible Retrieval Enhancements

Amazon S3 Glacier Flexible Retrieval has significantly improved data restore times, reducing them by up to 85%, and this improvement comes at no additional cost. These faster data restores are automatically applied to the Standard retrieval tier when Amazon S3 Batch Operations are used. Objects begin to be returned within minutes of initiating the restore process, facilitating quicker processing of restored data.

This enhancement benefits various tasks, such as transcoding media, restoring operational backups, training machine learning models, and analysing historical data. Leveraging S3 Batch Operations, archived data can be efficiently restored on a large scale by showing the objects to be retrieved and specifying the retrieval tier. The Standard retrieval tier now initiates restoring things within minutes, a substantial improvement compared to the previous 3–5 hour wait time. Overall restore throughput is also enhanced, thanks to new performance optimisations, enabling faster data workflows and quicker responses to business requirements.

AWS Backup Audit Manager's Delegated Backup Administrator Support

With AWS Organizations, the delegation of backup management has been expanded beyond the management account to dedicated administration accounts. This means that delegated administrators can now assume backup management responsibilities. As part of this launch, delegated backup administrators can centralise report generation and management on a large scale through AWS Backup Audit Manager. This feature is designed for auditing and reporting on data protection posture compliance.

Project Quotas Support in Amazon FSx for Lustre

Amazon FSx for Lustre, a fully managed service for high-performance file systems, now supports project quotas. This update allows users to easily group multiple files or directories within the file system into distinct projects and monitor storage usage per-project basis. Project quotas are particularly valuable for storage administrators responsible for managing file systems serving multiple projects or teams. This feature ensures no project exceeds its designated storage capacity, helping maintain effective resource allocation and control within the file system.

Beinex+ AWS Offerings

Beinex is an AWS consulting partner, and we empower customers to host their BI solutions and much more on the cloud. Our cloud migration experts bring in best-in-class stability and reliability by understanding your business strategy and working closely with you to deploy AWS infrastructure as a service.

AWS's commitment to innovation ensures that its storage solutions will remain at the forefront of the industry, providing users with the cutting-edge tools and capabilities required to thrive in the digital age. Embrace AWS storage solutions' possibilities and stay ahead in your data management endeavours.

Beinex Consulting has been awarded as the Alteryx 2020 Partner of the Year, Middle East at the Alteryx Partner Summit and Awards virtual event along with 15 other winners from North America, LATAM, and EMEA. During the Alteryx Summit, ‘Your Road to Revenue’, Alteryx celebrated the achievements and commitment of their partners to the Alteryx business and its customers. Beinex Consulting was awarded on the level of engagement in the Alteryx partner program and its efforts around driving innovation, growing revenue, and empowering Alteryx customers to solve our world’s most pressing business and societal issues in the Middle East Region. Selected among top Middle East Alteryx partners, Beinex demonstrated excellence in delivering end-to-end analytics transformation services that revolutionised multiple industries in the Middle East. Beinex Consulting Founder and Managing Director, Indumon Das indicates further growth for the digital transformation organisation soon: “Beinex continues to make strategic investments to enhance our association with Alteryx and clients in major Middle East markets. This award is a recognition to our continuous growth strategy and focus to be the best Middle East partner” “Through their ongoing pledge to the Alteryx Partner Program, our partners have demonstrated their commitment to helping Alteryx customers break down barriers and deliver game-changing insights.” – Josh Lewis, VP, Global Channels, Alteryx About Beinex Consulting Beinex is a digital transformation organization with a broad range of analytics modernization and training services. As a pioneer in analytics and cloud transformation, Beinex’s mission is to transform the way individuals and the organizations work with the data through innovation and experience. Beinex offers a broad range of robust and scalable business intelligence and analytics services to drive effective decision-making and create business value.

We are thrilled to recognize Beinex Consulting for being named Alteryx Middle East Partner of the Year!https://t.co/xwmp7HbsMp#TogetherWeSolve pic.twitter.com/4zic9mdlgD

— Alteryx (@alteryx) October 1, 2020

The brick-and-mortar banking model faced an existential threat with the emergence of Fintech (financial technology), or new technology that aims to enhance and automate the delivery and use of financial services. And later, new-gen technology companies started to deliver these services on secure digital platforms at a lower cost, resulting in traditional banks adopting advanced technology too.

Most banks were apprehensive about adopting automation to the finance function. This concern is frequently centered on whether it is possible to replace existing systems entirely with automation. The core banking system is perhaps the best example. Automating ‘untouchable’ core functions is not necessary. Instead, it can deal with the issues that surround them. But rooting your digital transformation in all other banking processes in intelligent, digital workflows is feasible.

What is digital banking?

Digital banking involves digitising all traditional banking products, procedures, and activities to serve customers through online channels. Examples include obtaining bank statements, cash withdrawals, funds’ transfers, accounts management and checking opening deposit accounts, loan management, bill payment, cheque management, and transaction records monitoring.

With digital banking, all banking services are accessible round-the-clock on mobile phones, PCs, and other intelligent devices. Thanks to digital banking software, all traditional services are now easier to obtain, comprehend, and manage.

Leading banks in the UAE make significant financial investments in the digital transformation of their banking operations and launch digital-only banks in the country.

Pros of digital banking

Here are some of the most known benefits of digital banking:

1. Scalability

Numerous functions provided by digital banks are just absent from traditional banks. This includes investing directly in stock markets and acquiring cryptocurrency and gold using the banking app. The user of digital banking can modify their security preferences, transaction caps, and even whether they wish to enable NFC or magnetic stripe transactions.

2. Personalisation

In digital banking, sophisticated personalisation tactics are powered by artificial intelligence (AI) and machine learning (ML). Customers can get timely financial solutions, interactive tools, and instructional resources from banks. Automatic budgeting, expenditure analytics, and savings reminders are a few technologies that can inform and engage customers.

3. Cost savings

Traditional banks spend a lot of time and expense on the checking and accounting processes. The elimination of unnecessary operations is what makes digital banking software's operational costs less expensive. With digital banking systems, banks may take on less labour by automating the procedures related to routine transactions. Because fewer persons and stages are required for transactions when technology is used, there is a lower chance of financial mistakes, and money transfers are more straightforward.

The touch of automation

Let’s also look at the most automated processes in the banking industry that have undergone complete digitization with the touch of automation.

1. Loan processing

RPA or Robotic Process Automation can reduce lengthy procedures that typically take months to as little as 10–15 minutes. Automation enables essential data extracted from customer-submitted documents to validate all details. Systems employ Machine Learning to make better-informed judgments based on data analytics supported by more straightforward statistical methods. Intermediary bots infer business logic and ask the user to correct any inaccurate entries to ensure safer loan judgments and automatic confirmation letter creation

2. Know Your Customer (KYC)

Not only is Know Your Customer (KYC) a necessary compliance procedure for any bank, but it is also the trickiest. To execute the customer checks, this process requires at least 150 and maybe thousands of FTEs.

According to Thomson Reuters, a small number of banks annually invest at least $500 million in KYC compliance. Banks have recently begun utilising RPA to gather and flawlessly check consumer information to cut costs and resources. Because of this, banks can now complete the KYC procedure with fewer resources and mistakes.

3. Anti-Money Laundering (AML)

One of the most data-intensive processes, AML, can be made simpler with some help from Robotic Process Automation (RPA). Implementing RPA has proven more efficient than labour-intensive traditional banking solutions in identifying suspicious banking transactions or automating repetitive operations.

4. Fraud Detection

Banks are concerned about enhancing their fraud detection system due to the rising banking fraud scenario. Banking fraud has increased since the introduction of cutting-edge technologies. Considering this, it is virtually hard for banks to manually review each transaction to spot fraud trends in real-time. RPA cleverly uses the "if-then" principle to spot any potential fraud and report it for prompt resolution with the relevant department.

5. Mortgage processing

Both banks and their clients find mortgage processing extremely labour-intensive and cumbersome. Before processing each loan request, banks manage their mortgage procedure for more than a month, going through several unnerving stages like job verification, credit checks, and inspection. The processing of mortgage loans could be severely delayed by even the slightest mistake made by either the customer or the bank.

But RPA has accelerated this process for banks. Robotics goes through a defined set of rules to eliminate potential bottlenecks and speed up mortgage processing.

Customer acceptance of digital banking services has rapidly increased throughout the UAE. According to bankers, the older generation, initially apprehensive, is now quickly adjusting, whereas younger clients are quick to accept digital offerings and digital-only platforms. Digital banks are drawing a lot of new consumers due to lower account maintenance expenses and improved deposit returns.

How Beinex can help you

Beinex is a Dubai-based digital transformation organisation anchored in innovation, creativity, and unrivalled customer service. Our extensive analytics modernisation and training services will empower you to construct an intelligent and dynamic banking enterprise. Our extensively experienced consultants enable a seamless digital experience for the banking industry.