Java對(duì)接ansible自動(dòng)運(yùn)維化平臺(tái)方式

更新時(shí)間：2023年04月04日 08:54:43 作者：猛男的溫柔

這篇文章主要介紹了Java對(duì)接ansible自動(dòng)運(yùn)維化平臺(tái)方式，具有很好的參考價(jià)值，希望對(duì)大家有所幫助。如有錯(cuò)誤或未考慮完全的地方，望不吝賜教

Java對(duì)接ansible自動(dòng)運(yùn)維化平臺(tái)實(shí)現(xiàn)文件采集分發(fā)
場(chǎng)景說(shuō)明及ansible yum安裝
Java代碼實(shí)現(xiàn)文件分發(fā)
POI創(chuàng)建文件工具類
創(chuàng)建主機(jī)組配置文件
實(shí)現(xiàn)文件分發(fā)
文件采集
總結(jié)

Java對(duì)接ansible自動(dòng)運(yùn)維化平臺(tái)實(shí)現(xiàn)文件采集分發(fā)

經(jīng)過(guò)大量查閱，網(wǎng)上使用Java對(duì)接ansible自動(dòng)運(yùn)維化平臺(tái)的示例代碼幾乎沒(méi)有，為了方便自己后期鞏固以及有需要的小伙伴，特以記錄?。?！

此次對(duì)接主要為以下兩個(gè)功能：

文件采集(對(duì)文件進(jìn)行批量操作，包括批量從多臺(tái)主機(jī)中采集共性文件如日志文件)
文件分發(fā)(對(duì)文件進(jìn)行批量操作，包括批量從多臺(tái)主機(jī)中分發(fā)共性文件如日志文件)

場(chǎng)景說(shuō)明及ansible yum安裝

因ansible沒(méi)有Windows的安裝包，所以為了方便測(cè)試，搭建了一套Linux環(huán)境進(jìn)行后續(xù)工作。

此次采用yum方式安裝，在采用yum方式安裝Ansible,首先安裝EPEL源。

yum install -y http://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

查看EPEL源中的Ansible版本

yum info ansible

直接安裝此版本，如果有其他要求，請(qǐng)調(diào)整源，安裝其他ansible版本

yum install -y ansible

安裝完成之后，查看ansible版本信息

ansible --version

配置Ansible服務(wù)器清單

清單文件/etc/ansible/hosts,在此文件中編寫節(jié)點(diǎn)主機(jī)的對(duì)應(yīng)IP地址和端口

我這里只是做一個(gè)演示，其中IP后面可以添加節(jié)點(diǎn)真實(shí)的SSH的端口，在定義的內(nèi)容上面有一個(gè)[]列表，里面的內(nèi)容為自定義內(nèi)容，方面為了操作綁定的節(jié)點(diǎn)主機(jī)，我習(xí)慣稱之為分組列表

簡(jiǎn)單的認(rèn)證一下，Ping一下添加的主機(jī)

成功安裝ansible ！！

Java代碼實(shí)現(xiàn)文件分發(fā)

顧名思義，文件分發(fā)就是把本機(jī)的文件分發(fā)到多個(gè)主機(jī)。

這時(shí)候就需要 Apache POI(大家可以去導(dǎo)入對(duì)應(yīng)的包)來(lái)創(chuàng)建本機(jī)的文件了(ansible Host配置文件也通過(guò)POI創(chuàng)建)

POI創(chuàng)建文件工具類

package com.tiduyun.cmp.operation.utils;

import com.tiduyun.cmp.common.model.operation.HostInfo;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.lang.StringUtils;
import org.springframework.stereotype.Component;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;

/**
 * @author huyuan@tiduyun.com ansible創(chuàng)建文件
 */
@Slf4j
@Component
public class AnsibleCreateFileUtils {
    private final static String filename = "hosts";

    public static String passWordConnect(List<HostInfo> hostInfo, String hostGroup , String directory) throws IOException{
        /** 在本地新建一個(gè)文件夾 里面創(chuàng)建一個(gè)文件 向里面寫入內(nèi)容 */


        // 創(chuàng)建文件夾對(duì)象 創(chuàng)建文件對(duì)象
        File folder = new File(directory);
        // 如果文件夾不存在 就創(chuàng)建一個(gè)空的文件夾
        if (!folder.exists()) {
            log.info("創(chuàng)建了文件夾{}" , folder);
            folder.mkdirs();
        }
        File file = new File(directory, filename);
        // 如果文件不存在 就創(chuàng)建一個(gè)空的文件
        if (!file.exists()) {
            try {
                log.info("創(chuàng)建了文件{}" , file);
                file.createNewFile();
            } catch (IOException e) {
                log.error("error data{}" , e);
            }
        }
        // 寫入數(shù)據(jù)
        // 創(chuàng)建文件字節(jié)輸出流
        FileOutputStream fos = new FileOutputStream(file);
        try {
            List<String> list = new ArrayList<>();
            for (HostInfo data : hostInfo) {
                // 開始寫
                String string = data.getHost() + " ansible_ssh_pass=" + data.getPasswd() + " ansible_ssh_user="
                    + data.getAccount() + " ansible_ssh_port=" + data.getPort();
                list.add(string);
            }
            String splicingData = StringUtils.join(list, "\n");
            String str = "[" + hostGroup + "]" + "\n" + splicingData;
            byte[] bytes = str.getBytes();
            // 將byte數(shù)組中的所有數(shù)據(jù)全部寫入
            fos.write(bytes);
            fos.flush();
            log.info("文件內(nèi)容{}" , str);
            // 刪除文件
            // deleteFile(file);
            // 關(guān)閉流

        } catch (IOException e) {
            log.error("error data{}" , e);
            throw e;
        }finally {
            if (fos != null) {
                fos.close();
            }
        }
        return directory;
    }

    public static void deleteFile(File file) {
        if (file.exists()) {// 判斷路徑是否存在
            if (file.isFile()) {// boolean isFile():測(cè)試此抽象路徑名表示的文件是否是一個(gè)標(biāo)準(zhǔn)文件。
                file.delete();
            } else {// 不是文件，對(duì)于文件夾的操作
                    // 保存 路徑D:/1/新建文件夾2 下的所有的文件和文件夾到listFiles數(shù)組中
                File[] listFiles = file.listFiles();// listFiles方法：返回file路徑下所有文件和文件夾的絕對(duì)路徑
                for (File file2 : listFiles) {
                    /*
                     * 遞歸作用：由外到內(nèi)先一層一層刪除里面的文件 再?gòu)淖顑?nèi)層 反過(guò)來(lái)刪除文件夾
                     *    注意：此時(shí)的文件夾在上一步的操作之后，里面的文件內(nèi)容已全部刪除
                     *         所以每一層的文件夾都是空的  ==》最后就可以直接刪除了
                     */
                    deleteFile(file2);
                }
            }
            file.delete();
        } else {
            log.error("該file路徑不存在??！");
        }

    }
}

創(chuàng)建主機(jī)組配置文件

注：ansible分為兩種連接方式，這里采用的是密鑰連接，生成的文件已拼接密鑰！！！后續(xù)的采集與分發(fā)都要用到這個(gè)。（如有不懂的小伙伴，可以去查找一下ansible的連接方式）

    @Override
    public void ansibleCreateHost(HostInfo hostInfo, String Key) {
        ParamCheckUtils.notNull(hostInfo, "hostInfo");

        List<HostInfo> HostIp = Arrays.asList(hostInfo);
        for (HostInfo data : HostIp) {
            String ansiblePassWd = data.getPasswd();
            String PassWd = hostInfoService.decode(ansiblePassWd);
            data.setPasswd(PassWd);
        }
        try {
            AnsibleCreateFileUtils.passWordConnect(HostIp, ansibleConfigurationItemVo.getHostGroup(),
                ansibleConfigurationItemVo.getDirectory());
        } catch (IOException e) {
            log.error("Failed to create host configuration{}", e);
        }
    }

實(shí)現(xiàn)文件分發(fā)

主機(jī)配置文件已經(jīng)配置好，接下來(lái)就是執(zhí)行ansible對(duì)應(yīng)的命令，通過(guò)Java拼接ansible命令。

執(zhí)行命令工具類

package com.tiduyun.cmp.operation.utils;

import lombok.extern.slf4j.Slf4j;

import java.io.BufferedReader;
import java.io.IOException;
import java.io.InputStreamReader;
import java.nio.charset.Charset;

import static cn.hutool.db.DbUtil.close;

/**
 * @author huyuan@tiduyun.com ansible執(zhí)行命令工具類
 * upload 上傳文件
 * createRemoteDirectory  創(chuàng)建遠(yuǎn)程目錄
 */
@Slf4j
public class AnsibleExecuteTheOrderUtils {

    private final static String commandBin = "/bin/sh";

    private final static String commandC = "-c";


    /**
     *  創(chuàng)建遠(yuǎn)程目錄
     */
    public static void createRemoteDirectory(String hostGroup, String remotePath, String directory) throws IOException {
        Runtime run = Runtime.getRuntime();
        String[] cmds = new String[3];
        cmds[0] = commandBin;
        cmds[1] = commandC;
        cmds[2] =
                "ansible " + hostGroup + " -m command -a " + "\"mkdir " + remotePath + "\"" + " -i " + directory + "/hosts";

        // 執(zhí)行CMD命令
        Process p = run.exec(cmds);
        log.info("ansible遠(yuǎn)程執(zhí)行命令為{}", cmds[2]);

        BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8")));
        try {
            String lineMes;
            while ((lineMes = br.readLine()) != null)
                log.info(lineMes);// 打印輸出信息
            try {
                // 檢查命令是否執(zhí)行失敗。
                if (p.waitFor() != 0) {
                    if (p.exitValue() == 1)// 0表示正常結(jié)束，1：非正常結(jié)束
                        log.error("命令執(zhí)行失敗");
                }
            } catch (InterruptedException e) {
                log.error("error data{}", e);
            }
        } catch (IOException e) {
            log.error("fail to carry out command{}", e);
            throw e;
        } finally {
            if (br != null) {
                br.close();
            }
        }

    }

    /**
     *  文件分發(fā)
     */
    public static void upload(String hostGroup, String localPath, String remotePath, String directory)
        throws IOException {
        Runtime run = Runtime.getRuntime();
        String[] cmds = new String[3];
        cmds[0] = commandBin;
        cmds[1] = commandC;
        cmds[2] = "ansible " + hostGroup + " -m copy -a " + "\"src=" + localPath + " dest=" + remotePath + "\"" + " -i "
            + directory + "/hosts";
        // 執(zhí)行CMD命令
        Process p = run.exec(cmds);
        log.info("ansible命令為{}", cmds[2]);

        BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8")));
        try {
            String lineMes;
            while ((lineMes = br.readLine()) != null)
                log.info("ansible輸出信息為 :" + lineMes);// 打印輸出信息
            try {
                // 檢查命令是否執(zhí)行失敗。
                if (p.waitFor() != 0) {
                    if (p.exitValue() == 1)// 0表示正常結(jié)束，1：非正常結(jié)束
                        log.error("命令執(zhí)行失敗");
                }
            } catch (InterruptedException e) {
                log.error("error data{}", e);
            }
        } catch (IOException e) {
            log.error("fail to carry out command{}", e);
            throw e;
        } finally {
            if (br != null) {
                br.close();
            }
        }

    }

    /**
     *  文件采集
     */
    public static void fileCollection(String hostGroup, String remotePath, String localPath , String directory) throws IOException {
        Runtime run = Runtime.getRuntime();
        String[] cmds = new String[3];
        cmds[0] = commandBin;
        cmds[1] = commandC;
        cmds[2] = "ansible " + hostGroup + " -m fetch -a " + "\"src=" + remotePath + " dest=" + localPath + " force=yes backup=yes\"" + " -i "
                + directory + "/hosts";

        // 執(zhí)行CMD命令
        Process p = run.exec(cmds);
        log.info("ansible遠(yuǎn)程采集文件命令為{}", cmds[2]);

        BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8")));
        try {
            String lineMes;
            while ((lineMes = br.readLine()) != null)
                log.info(lineMes);// 打印輸出信息
            try {
                // 檢查命令是否執(zhí)行失敗。
                if (p.waitFor() != 0) {
                    if (p.exitValue() == 1)// 0表示正常結(jié)束，1：非正常結(jié)束
                        log.error("命令執(zhí)行失敗");
                }
            } catch (InterruptedException e) {
                log.error("error data{}", e);
            }
        } catch (IOException e) {
            log.error("fail to carry out command{}", e);
            throw e;
        } finally {
            if (br != null) {
                br.close();
            }
        }

    }

    public static void ExecuteTheOrder(String command) throws IOException {
        log.info("start execute cmd {}", command);

        String[] cmd = new String[] {"/bin/bash", "-c", command};
        Runtime run = Runtime.getRuntime();
        Process p = run.exec(cmd); // 執(zhí)行CMD命令

        BufferedReader br = new BufferedReader(new InputStreamReader(p.getInputStream(), Charset.forName("UTF-8")));
        try {
            String lineMes;
            while ((lineMes = br.readLine()) != null)
                log.info("輸出信息為 {}", lineMes);// 打印輸出信息
            try {
                // 檢查命令是否執(zhí)行失敗。
                if (p.waitFor() != 0) {
                    if (p.exitValue() == 1)// 0表示正常結(jié)束，1：非正常結(jié)束
                        log.error("命令執(zhí)行失敗");
                }
            } catch (InterruptedException e) {
                log.error("error data{}", e);
            }

        } catch (IOException e) {
            log.error("fail to carry out command{}", e);
            throw e;
        } finally {
            if (br != null) {
                br.close();
            }
        }
    }

    public static void disconnect() {
        try {
            close();
        } catch (Exception ex) {
            // Ignore because disconnection is quietly
        }
    }

    // public void execute(String command) throws Exception {
    // log.info("start execute cmd {}", command);
    // try (Session session = sshClient.startSession()) {
    // Session.Command exec = session.exec(command);
    //
    // Integer readLineCount = 0;
    // InputStream in = exec.getInputStream();
    // log.info(IOUtils.readFully(in).toString());
    // String errorMessage = IOUtils.readFully(exec.getErrorStream(), LoggerFactory.DEFAULT).toString();
    // log.info(errorMessage);
    // if (exec.getExitStatus() != null && exec.getExitStatus() != 0) {
    // throw new RuntimeException(
    // "exec " + command + " error,error message is " + errorMessage + ",error code " + exec.getExitStatus());
    // }
    // log.info("exec result code {}", exec.getExitStatus());
    //
    // }
    //
    // }
}

接下來(lái)就是調(diào)用

package com.tiduyun.cmp.operation.service.impl;

import cn.hutool.core.util.RandomUtil;
import cn.hutool.core.util.StrUtil;
import com.tiduyun.cmp.common.model.flow.UploadFile;
import com.tiduyun.cmp.common.model.operation.ComponentInfo;
import com.tiduyun.cmp.common.model.operation.HostInfo;
import com.tiduyun.cmp.common.provider.service.ExceptionBuildService;
import com.tiduyun.cmp.operation.constant.OperationExceptionCode;
import com.tiduyun.cmp.operation.constant.StartCmdSeparate;
import com.tiduyun.cmp.operation.model.AnsibleConfigurationItemVo;
import com.tiduyun.cmp.operation.model.vo.FileQueryVo;
import com.tiduyun.cmp.operation.service.AnsibleTaskRecordService;
import com.tiduyun.cmp.operation.service.ComposerDeployService;
import com.tiduyun.cmp.operation.service.HostInfoService;
import com.tiduyun.cmp.operation.service.UploadFileService;
import com.tiduyun.cmp.operation.utils.AnsibleExecuteTheOrderUtils;
import lombok.extern.slf4j.Slf4j;
import org.apache.commons.collections4.CollectionUtils;
import org.apache.commons.lang3.StringUtils;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.stereotype.Service;

import java.io.File;
import java.util.ArrayList;
import java.util.Arrays;
import java.util.List;

@Slf4j
@Service
public class AnsibleDeployServiceImpl implements ComposerDeployService {

    @Value(value = "${cmp.operation.commandHeader:cmd /c}")
    private String commandHeader;

    @Value(value = "${cmp.operation.filePath:/data/cmp/file}")
    private String filePath;

    @Value(value = "${cmp.operation.remoteFilePath:/tmp}")
    private String remoteFilePath;

    @Autowired
    private AnsibleTaskRecordService ansibleTaskRecordService;

    @Autowired
    private AnsibleConfigurationItemVo ansibleConfigurationItemVo;

    @Autowired
    private UploadFileService uploadFileService;

    @Autowired
    private HostInfoService hostInfoService;

    @Autowired
    private ExceptionBuildService exceptionBuildService;

    @Override
    public void deploy(HostInfo hostInfo, ComponentInfo componentInfo, String cpmposerName) {
        ansibleTaskRecordService.ansibleCreateHost(hostInfo, null);

        try {
            String remotePath = StringUtils.join(remoteFilePath, "/", cpmposerName, "-", componentInfo.getName(), "-",
                RandomUtil.randomString(3));
            log.info("remote file path = {}", remotePath);

            List<Integer> fileIds = getFileIds(componentInfo.getFileUrl());
            if (CollectionUtils.isNotEmpty(fileIds)) {
                FileQueryVo uploadFileQueryVo = new FileQueryVo();
                uploadFileQueryVo.setIds(fileIds);
                List<UploadFile> uploadFiles = uploadFileService.query(uploadFileQueryVo);
                for (UploadFile uploadFile : uploadFiles) {
                    String path = StringUtils.join(filePath, uploadFile.getFilePath());
                    File file = new File(path);
                    if (!file.exists()) {
                        log.error("file url is {}", file.getPath());
                        throw exceptionBuildService.buildException(OperationExceptionCode.FILE_NOT_EXIST,
                            new Object[] {uploadFile.getFileName()});
                    }
                    // 創(chuàng)建遠(yuǎn)程目錄
                    AnsibleExecuteTheOrderUtils.createRemoteDirectory(ansibleConfigurationItemVo.getHostGroup(),
                        StringUtils.join(remotePath), ansibleConfigurationItemVo.getDirectory());

                    // 分發(fā)文件
                    AnsibleExecuteTheOrderUtils.upload(ansibleConfigurationItemVo.getHostGroup(), path,
                        StringUtils.join(remotePath, "/", uploadFile.getFileName()),
                        ansibleConfigurationItemVo.getDirectory());
                }
            }
            List<String> startCmds = getStartCmds(componentInfo.getStartCmd());
            if (CollectionUtils.isNotEmpty(startCmds)) {
                String cdCmd = StringUtils.join("cd ", remotePath);
                String execCmd = StringUtils.join(startCmds, ";");
                execCmd = StringUtils.join(cdCmd, ";", execCmd);
                log.info("execCmd= " + execCmd);
                // sshClient.execute(execCmd);
                AnsibleExecuteTheOrderUtils.ExecuteTheOrder(execCmd);

            } else {
                log.error("parse startCmd fail {}", componentInfo.getStartCmd());
            }

        } catch (Exception e) {
            log.error("主機(jī)[{}]部署[{}]組件失敗,主機(jī)ID[{}],組件ID[{}]:", hostInfo.getHost(), componentInfo.getName(),
                hostInfo.getId(), componentInfo.getId(), e);
            throw exceptionBuildService.buildException(OperationExceptionCode.EXECUTE_CMD_ERROR,
                new Object[] {e.getMessage()});

        } finally {
            AnsibleExecuteTheOrderUtils.disconnect();
        }

    }

    @Override
    public boolean isSupport(HostInfo hostInfo) {
        return true;
    }

    private List<Integer> getFileIds(String fileIds) {
        List<Integer> ids = new ArrayList<>();
        if (fileIds == null) {
            return null;
        }
        String[] split = StringUtils.split(fileIds, ",");
        for (String s : split) {
            ids.add(Integer.parseInt(s));
        }
        return ids;
    }

    private List<String> getStartCmds(String startCmd) {
        List<String> cmd = new ArrayList<>();
        if (startCmd == null) {
            return cmd;
        }
        String[] split = StrUtil.split(startCmd, StartCmdSeparate.SIGN);
        cmd.addAll(Arrays.asList(split));
        return cmd;

    }

    public static Boolean needCd(String s) {
        String[] splits = StrUtil.split(s, "&&");
        int maxIndex = splits.length - 1;
        String cmd = splits[maxIndex];
        if (StrUtil.startWith(cmd, "cd")) {
            return false;
        } else {
            return true;
        }

    }
}

文件采集

同上，調(diào)用兩個(gè)工具類

@Override
    public void fileCollection(HostInfo hostInfo, String remotePath, String localPath) {
        ansibleCreateHost(hostInfo, null);
        try {
            log.info("remote file path = {}", remotePath);
            log.info("local file path = {}", localPath);

            // 文件采集
            AnsibleExecuteTheOrderUtils.fileCollection(ansibleConfigurationItemVo.getHostGroup(), remotePath,
                localPath , ansibleConfigurationItemVo.getDirectory());
        } catch (Exception e) {
            log.error("主機(jī)[{}]文件采集失敗,主機(jī)ID[{}]:", hostInfo.getHost(), hostInfo.getId(), e);
            throw exceptionBuildService.buildException(OperationExceptionCode.EXECUTE_CMD_ERROR,
                new Object[] {e.getMessage()});

        } finally {
            AnsibleExecuteTheOrderUtils.disconnect();
        }

    }