《计算机应用研究》|Application Research of Computers

基于MapReduce和IFOA的并行密度聚类算法

Density-based clustering algorithm by using improve fruit fly optimization based on MapReduce

免费全文下载 (已被下载 次)  
获取PDF全文
作者 胡健,徐锴滨,毛伊敏
机构 1.江西理工大学 信息工程学院,江西 赣州 343100;2.江西理工大学应用科学学院 信息工程系,江西 赣州 341000
统计 摘要被查看 次,已被下载
文章编号 1001-3695(2021)05-010-1336-08
DOI 10.19734/j.issn.1001-3695.2020.08.0187
摘要 针对大数据下密度聚类算法中存在的数据划分不合理、参数寻优能力不佳、并行性能较低等问题,提出一种基于IFOA的并行密度聚类算法(density-based clustering algorithm by using improve fruit fly optimization based on MapReduce,MR-DBIFOA)。首先,该算法基于KD树,提出网格划分策略(divide gird based on KD tree,KDG)来自动划分数据网格;其次在局部聚类中,提出基于自适应搜索策略(step strategy based on knowledge learn,KLSS)和聚类判定函数(clustering criterion function,CCF)的果蝇群优化算法(improve fruit fly optimization algorithm,IFOA);然后根据IFOA进行局部聚类中最优参数的动态寻优,从而使局部聚类的聚类效果得到提升;同时结合MapReduce模型提出局部聚类算法DBIFOA(density-based clustering algorithm using IFOA);最后提出了基于QR-tree的并行合并局部簇算法(cluster merging algorithm by using MapReduce,MR-QRMEC),实现局部簇的并行合并,使算法整体的并行性能得到加强。实验表明,MR-DBIFOA在大数据下的并行效率更高,且聚类效果更好。
关键词 大数据; 密度聚类算法; KD树; 果蝇优化
基金项目 国家重点研发计划资助项目(2018YFC1504705)
国家自然科学基金资助项目(41562019)
江西省教育厅科技项目(GJJ151528,GJJ151531)
本文URL http://www.netgaindomains.com/article/01-2021-05-010.html
英文标题 Density-based clustering algorithm by using improve fruit fly optimization based on MapReduce
作者英文名 Hu Jian, Xu Kaibin, Mao Yimin
机构英文名 1.School of Information Engineering,Jiangxi University of Science & Technology,Ganzhou Jiangxi 343100,China;2.Dept. of Information Engineering,College of Applied Science,Jiangxi University of Science & Technology,Ganzhou Jiangxi 341000,China
英文摘要 Aiming at the problems of unreasonable division of data gridding, poor parameter optimization ability and low efficiency of parallelization in big data density-based clustering algorithm, this paper proposed a density-based clustering algorithm by using improve fruit fly optimization based on MapReduce, named MR-DBIFOA. Firstly, based on KD-Tree, it designed a division strategy to divide the cell of grid adaptively. Secondly, this method proposed an improve fruit fly optimization algorithm which used KLSS and CFF. Then, based on IFOA algorithm, it dynamically selected the optimal parameters of local clustering, which could improve the clustering effect of local clustering. Meanwhile, in order to improve the parallel efficiency, it proposed a density-based clustering algorithm using IFOA to parallel compute the local clusters of clustering algorithm. Finally, based on QR-Tree and MapReduce, it proposed a clusters merging algorithm(MR-QRMEC) to get the result of clustering algorithm more quickly, which improved the core clusters merging efficiency of density-based clustering algorithm. The experimental results show that the MR-DBIFOA has better clustering results and performs better parallelization in big data.
英文关键词 big data; density-based clustering algorithm; KD-tree; fruit fly optimization
参考文献 查看稿件参考文献
 
收稿日期 2020/8/5
修回日期 2020/9/14
页码 1336-1343
中图分类号 TP315.69
文献标志码 A
天下色情网