One post tagged with "tdsql"

In the wave of digital transformation across the financial industry, distributed databases have become the preferred choice for financial institutions building core business systems due to their high scalability, availability, and performance. TDSQL, the distributed database launched by Tencent Cloud, is widely used in finance, internet, and government sectors. However, with continuous business growth and explosive data expansion, optimizing TDSQL database performance has become a challenge for many enterprises and developers. This article introduces PawSQL's specialized optimization guide for TDSQL databases, helping users fully leverage TDSQL's performance potential.

I. The Importance of TDSQL Database Performance Optimization​

TDSQL databases face demanding challenges in financial applications, including high concurrency, large data volumes, and high reliability requirements. Performance optimization affects not only business response speed and user experience but also directly impacts financial institutions' operational efficiency and risk control capabilities. By optimizing TDSQL databases, users can achieve the following objectives:

• Improve Business Response Speed: Reduce query latency, increase transaction processing speed, and provide users with a smoother service experience.
• Lower Operational Costs: Optimize resource utilization, reduce hardware investment and maintenance costs, and improve economic benefits.
• Enhance System Stability: Improve database stability and reliability through reasonable optimization strategies, reducing the risk of system failures.

II. PawSQL's Specialized Optimization Capabilities for TDSQL​

1. Deep SQL Syntax Support for TDSQL​

PawSQL provides deep SQL syntax support for TDSQL databases, helping users better leverage TDSQL's features for performance optimization.

• Complete Support for MySQL Syntax: Since TDSQL is developed based on MySQL, PawSQL fully supports the MySQL syntax system, ensuring that users can seamlessly work with TDSQL's syntax requirements when using PawSQL for SQL optimization.
• Complete Parsing of TDSQL-Specific DDL Syntax: TDSQL has some unique DDL syntax, such as creating distributed tables and defining partitioned tables. PawSQL can fully parse these special syntaxes and provide accurate SQL optimization recommendations.

-- hash sharding or broadcast table
CREATE TABLE [IF NOT EXISTS] tbl_name
    [(create_definition)]
    [local_table_options]
shardkey=column_name|noshardkey_allset
-- range or list sharding
CREATE TABLE [IF NOT EXISTS] tbl_name
    [(create_definition)]
    [local_table_options]
TDSQL_DISTRIBUTED BY range|list (column_name) [partition_options]

2. Enhanced Optimization Rule System​

PawSQL has added multiple optimization rules targeting TDSQL's distributed characteristics to help users avoid common performance issues.

• Distributed SQL Design Standards:

  • Avoid Table Joins on Non-Shard Keys: In distributed databases, joining tables on fields that are not shard keys causes cross-node data transmission, increasing network overhead. PawSQL reminds users to use shard keys as join fields whenever possible.
  • Avoid Table Joins in Distributed Database DML: In distributed environments, DML operations involving table joins may lead to complex distributed transaction processing, affecting performance. PawSQL recommends users avoid table joins in DML operations.
  • Distributed Database DML Missing Equality Conditions on Shard Fields: In distributed databases, DML operations without equality conditions on shard fields lead to full table scans, severely impacting performance. PawSQL reminds users to always include equality conditions on shard fields in DML operations.

• Shard Key Design Standards:

  • Not Recommended to Use Multiple Fields as Shard Keys: Using multiple fields as shard keys may cause uneven data distribution and increase query complexity. PawSQL recommends using a single field as the shard key when possible.
  • Use High-Cardinality Fields as Shard Keys: The higher the cardinality of shard keys, the more evenly data is distributed, resulting in better query performance. PawSQL reminds users to select high-cardinality fields as shard keys.

• Distribution Strategy Design Standards:

  • Large Tables Should Not Use Replication Distribution: Using replication distribution for large tables causes data storage redundancy, increasing storage costs and network overhead. PawSQL recommends using hash or range distribution for large tables.
  • Hash Distribution is Recommended: Hash distribution achieves uniform data distribution and improves query performance. PawSQL recommends prioritizing hash distribution.
  • Avoid Non-Distributed Tables: Non-distributed tables in distributed databases may lead to centralized data storage, affecting system scalability and performance. PawSQL reminds users to avoid using non-distributed tables whenever possible.

III. SQL Optimization Full Lifecycle Matrix​

1. Development and Testing Phase: Intelligent SQL Optimization​

During the development and testing phase, PawSQL provides a one-stop online SQL optimization tool for application developers and testers.

• Query Rewriting Optimization: PawSQL automatically rewrites SQL queries, such as converting complex subqueries into more efficient join queries and optimizing the order of query conditions, helping users improve query performance.
• Intelligent Index Recommendations: Based on SQL query characteristics and data distribution, PawSQL provides intelligent index recommendations. Creating appropriate indexes can significantly improve query speed and reduce data scan volume.

The PawSQL optimization platform has been integrated with popular IDEs (VSCode and JetBrains), allowing developers to optimize SQL without leaving their development environment, improving work efficiency.

2. Code Integration Phase: Comprehensive SQL Auditing​

During the code integration phase, PawSQL's auditing platform provides comprehensive and precise intelligent SQL auditing capabilities for SQL quality management teams through its leading core technologies.

• Proprietary SQL Parser: PawSQL's proprietary SQL parser accurately parses various complex SQL statements, providing precise syntactic information for subsequent rule matching and optimization recommendations.
• Syntax Tree-Based Rule Matching: PawSQL performs in-depth analysis of SQL statements by constructing SQL syntax trees, matching various optimization rules, and ensuring the accuracy of audit results.
• Contextual Information Updates: PawSQL dynamically updates audit results based on the contextual information of SQL statements, providing optimization recommendations that better match actual situations.

PawSQL conducts comprehensive checks across syntax standards, performance efficiency, security, and other dimensions, providing targeted optimization suggestions to help enterprises improve SQL performance and application efficiency. For TDSQL's distributed characteristics, PawSQL offers specialized distributed query optimization recommendations, with applicable rule data exceeding 190 rules.

3. Operations Phase: Performance Inspection Platform​

During the operations phase, PawSQL's database performance inspection platform automatically captures slow queries generated in the database regularly and provides SQL optimization recommendations.

• Slow Query Inspection: PawSQL automatically captures slow query statements in the database, analyzes their execution plans and performance bottlenecks, and provides detailed optimization recommendations. By optimizing slow queries, users can significantly improve overall database performance.
• Database Object Inspection: PawSQL also regularly inspects database objects to identify potential performance, security, maintainability, and other issues, providing optimization recommendations. For example, it checks index usage, table space utilization, and other factors, helping users detect and resolve potential problems promptly.

PawSQL supports slow query inspection and database object inspection for TDSQL databases, providing comprehensive performance monitoring and optimization support for operations personnel.

IV. Conclusion​

PawSQL for TDSQL databases provides users with a one-stop performance optimization solution. From intelligent SQL optimization during development and testing to comprehensive SQL auditing during code integration and the performance inspection platform during operations, PawSQL covers the entire lifecycle of database performance optimization. By using PawSQL, users can fully leverage the performance potential of TDSQL databases, improve business response speed, reduce operational costs, and enhance system stability.

In financial-grade distributed database applications, performance optimization is a long-term and challenging task. PawSQL will continue to monitor TDSQL database developments and user needs, continuously optimizing and improving the rule system to provide higher quality and more efficient performance optimization services. Experience PawSQL now to unlock TDSQL database performance potential and safeguard your business development!

Try it now: https://www.pawsql.com/app