Skip to search boxSkip to navigationSkip to main content

Implementation of the Spark technique in a matrix distributed computing algorithm

*Corresponding author for this work
  • Tianjin Maritime College
    ,
  • Trakya University
    ,
  • University of Fujairah
Research Output:
Contribution to journal
Article
Research
Peer-review

Open access

Abstract

Two analyzes of Spark engine performance strategies to implement the Spark technique in a matrix distributed computational algorithm, the multiplication of a sparse multiplication operational test model. The dimensions of the two input sparse matrices have been fixed to 30,000 × 30,000, and the density of the input matrix have been changed. The experimental results show that when the density reaches about 0.3, the original dense matrix multiplication performance can outperform the sparse-sparse matrix multiplication, which is basically consistent with the relationship between the sparse matrix multiplication implementation in the single-machine sparse matrix test and the computational performance of the local native library. When the density of the fixed sparse matrix is 0.01, the distributed density-sparse matrix multiplication outperforms the same sparsity but uses the density matrix storage, and the acceleration ratio increases from 1.88× to 5.71× with the increase in dimension. The overall performance of distributed operations is improved.

Research Output Information

Output type

Research Output:
Contribution to journal
Article
Research
Peer-review

Original language

English

Pages from-to (Number of pages)

Pages 660-671 (12 pages)

Journal (Volume, Issue Number)

Journal of Intelligent Systems (Volume 31, Issue 1)

Publication milestones

  • Published - 01/01/2022

Publication status

Published - 01/01/2022

ISSN

0334-1860

Publication IDs

  • Scopus: 85132334976

Publication metrics