Data processing logic hidden behind the LCD mosaic screen

Time:2023-02-13 Views:674

The large LCD mosaic screen is a simple screen on the surface, and the data processing logic behind the screen is sometimes very complex; Solution providers, system providers and large screen integrators, regardless of the role of installing the LCD mosaic screen, if the entire solution cannot solve the problem for users, the screen will not have the value of installation. Sometimes there are some strange problems that cannot be realized. For example, if the financial big data processing center can not monitor the data reasonably, it can be used directly, without considering the sensitivity of the data and the level of users, and the end of shift products will not be used.

（2） LCD splicing screen monitoring object management

Data quality monitoring objects are mainly divided into two categories: taxpayers and tax authorities. Different data quality monitoring dimensions and monitoring methods for monitoring objects are different. On the basis of collecting the monitoring needs of each business department of the bureau, the data underlying monitoring logic is set up to increase the types of data quality monitoring objects for use in quality rule configuration.

（2） Quality rule management

Configure corresponding quality rules according to data quality monitoring objects and quality monitoring purposes. Support user-defined quality detection models. The model types include column value detection, column value fluctuation detection, cross-table foreign key check and reference integrity check; In terms of data analysis monitoring rules, common analysis rules such as Boolean analysis, integrity analysis, standard value range analysis, enumeration value analysis, numerical analysis and uniqueness analysis are supported;

In terms of data analysis and repair rules, support includes removing white space characters, removing specific characters, case conversion, data desensitization, Chinese to pinyin, synonym replacement, ID card verification conversion, ID card information extraction, date splitting, string interception, conversion to string, conversion to Unix timestamp, full-width and half-width conversion, multi-table merging, multi-column merging, splitting column, data grouping and merging, null value filling, date standardization Phone number cleaning, zoning standardization, historical zip table, real-time business table and SQL cleaning.

（3） LCD splicing screen data quality monitoring

Track the whole process of data quality monitoring tasks, provide automatic and manual task monitoring methods, customize the task monitoring cycle, provide task effective cycle selection, enable and disable monitoring tasks in real time, obtain task monitoring results in real time, view the details of task monitoring results, and provide standard statistical dimensions, and provide customized statistical methods according to user needs.

（4） Data quality report

It provides the function of checking the null value rate and validity rate of data table fields, and can generate statistical data quality integrity and validity evaluation reports, and supports downloading. Integrity evaluation: The incomplete ratio refers to the proportion of null value fields, which refers to the ratio of incomplete columns in each table, i.e. the number of incomplete fields/the total number of fields. Validity rate of value range: number of legal values/total number of values

（5） Data quality improvement

Provide data quality problem processing process, customize processing task flow rules according to user needs, task processing terminal, provide visual LCD splicing screen data processing, drag and drop operation mode, and support common visual data processing methods such as removal of specific characters, case conversion, data desensitization, Chinese to pinyin, synonym conversion and ID card verification conversion.