快速搭建对象存储服务 - Minio,并解决临时地址暴露ip、短链接请求改变浏览器地址等问题
本文要解决的问题
基础的Minio下载安装、java操作方法、完整的工具类。
使用minio时需要注意的地方:
使用Minio的时候,生成资源的临时访问链接时,生成的地址IP是真实的IP和端口,不安全,怎么办?
生成的Minio的临时访问链接过长怎么办?
从而引导出:
1、如何生成短链接:
2、重定向和转发的区别?
3、重定向的实现方式:
4、如何保证浏览器地址不变的情况下请求资源?
基于Docker-compose的Minio安装
配置文件docker-compose.yml
创建docker-compose.yml,并在docker-compose.yml文件夹下面创建文件夹data
mkdir data
version: "3"
services:minio:image: minio/miniocontainer_name: minioprivileged: true #是否要获取宿主机的特权volumes:- /root/common/data/minio/data:/data #目录映射ports:- "9001:9001" #端口映射 ---- 不能采用:9001:9000之类的- "9000:9000" #端口映射environment:MINIO_ACCESS_KEY: xxxxxA1 #账号MINIO_SECRET_KEY: xxxxxA2 #密码command: server --console-address ':9001' /data #启动命令healthcheck: #健康检测test: ["CMD", "curl", "-f", "http://localhost:9000/minio/health/live"]interval: 30stimeout: 20sretries: 3
启动
docker-compose up -d # 后台启动
启动成功后登录webUI
URL:ip:9001
Access Key:xxxxxA1
Secret_key:xxxxxA2
上传资源-web操作
创建bucket
上传资源
上传成功后的样式
Java使用 --- 基础版
Step1、引入pom依赖
<dependency><groupId>io.minio</groupId><artifactId>minio</artifactId><version>8.5.17</version>
</dependency>
Step2、创建一个新key,并下载
此时系统会下载:credentials.json
{"url":"http://ip:9001/api/v1/service-account-credentials","accessKey":"xxxxx","secretKey":"xxxxxxx","api":"s3v4","path":"auto"}
Step3、创建一个application.yml文件
minio:url: "http://ip:9000" # 如果是http://ip:9001 会报错“S3 API Requests must be made to API port.”
# url: "http://www.abcxxxx域名.com" # 需要配置 指向MinIO真实地址:http://ip:9000; --- 可以使用Nginx反向代理 --- 解决暴露真实IP的方法accessKey: "xxxxxA1"secretKey: "xxxxxA2"api: "s3v4"
Step4、创建配置实体MinioProperties.java
import lombok.Data;
import org.springframework.boot.context.properties.ConfigurationProperties;@ConfigurationProperties(prefix = "minio")
@Data
public class MinioProperties {private String url;private String accessKey;private String secretKey;private String api;
}
Step5、创建文件类型枚举类FileType.java
package com.lting.minio.model;import lombok.Getter;
/**
* 枚举类,按需定义
*/
public enum FileType {// 修正后的枚举定义(包含扩展名和MIME类型)MP4("mp4", "video/mp4"),TXT("txt", "text/plain"),JSON("json", "application/json"),PNG("png", "image/png"), // 修正PNG类型JPG("jpg", "image/jpeg"),PDF("pdf", "application/pdf"),DOC("doc", "application/msword"),DOCX("docx", "application/vnd.openxmlformats-officedocument.wordprocessingml.document"),CSV("csv", "text/csv"),EXCEL("xls", "application/vnd.ms-excel");private final String extension;@Getterprivate final String mimeType;// 枚举构造函数FileType(String extension, String mimeType) {this.extension = extension;this.mimeType = mimeType;}// 根据文件名获取MIME类型public static String getMimeTypeForFile(String fileName) {String extension = getFileExtension(fileName);for (FileType type : values()) {if (type.extension.equalsIgnoreCase(extension)) {return type.mimeType;}}return "application/octet-stream"; // 默认类型}// 根据扩展名获取枚举实例public static FileType fromExtension(String extension) {for (FileType type : values()) {if (type.extension.equalsIgnoreCase(extension)) {return type;}}throw new IllegalArgumentException("Unsupported file extension: " + extension);}// 获取文件扩展名辅助方法private static String getFileExtension(String fileName) {int lastDotIndex = fileName.lastIndexOf('.');if (lastDotIndex == -1 || lastDotIndex == fileName.length() - 1) {return "";}return fileName.substring(lastDotIndex + 1).toLowerCase();}
}
Step6、创建工具类MinioUtils.java
package com.lting.minio;import com.lting.minio.model.FileType;
import com.lting.minio.model.MinioProperties;
import io.minio.*;
import io.minio.errors.*;
import io.minio.http.Method;
import io.minio.messages.Bucket;
import io.minio.messages.Item;
import jakarta.annotation.PostConstruct;
import lombok.Getter;
import lombok.extern.log4j.Log4j2;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Component;
import org.springframework.web.multipart.MultipartFile;import java.io.*;
import java.nio.charset.StandardCharsets;
import java.nio.file.Files;
import java.nio.file.Path;
import java.security.InvalidKeyException;
import java.security.NoSuchAlgorithmException;
import java.util.List;
import java.util.concurrent.TimeUnit;
import java.util.stream.Collectors;
import java.util.stream.StreamSupport;/*** 0、桶列表* 1、CRUD桶Buckets* 2、Upload(指定路径)* 3、查看桶下面的list列表* 4、Delete* 5、Download*/
@Component
@Log4j2
public class MinioUtils {@Getterprivate final MinioProperties minioProperties;private MinioClient minioClient;@Autowiredpublic MinioUtils(MinioProperties minioProperties) {this.minioProperties = minioProperties;}@PostConstructpublic void init() {try {this.minioClient = MinioClient.builder().endpoint(minioProperties.getUrl()) // 使用代理域名---不然生成的临时地址为真实ip地址.credentials(minioProperties.getAccessKey(), minioProperties.getSecretKey()).build();} catch (Exception e) {log.error("初始化minio配置异常", e.fillInStackTrace());}}/*** 已经存在的bucket集合列表*/public List<Bucket> listBucket() throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {return minioClient.listBuckets();}/*** 创建buckets*/public void createBuckets(final String bucketName) throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {minioClient.makeBucket(MakeBucketArgs.builder().bucket(bucketName).build());}/*** 某一个bucket是否存在*/public boolean bucketExists(final String bucketName) throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {return this.minioClient.bucketExists(BucketExistsArgs.builder().bucket(bucketName).build());}/*** 删除buckets*/public void removeBucket(final String bucketName) throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {minioClient.removeBucket(RemoveBucketArgs.builder().bucket(bucketName).build());}/*** 上传文件* 通用上传方法*/public ObjectWriteResponse uploadFile(final String bucketName, final String fileName, final InputStream inputStream, final long fileSize, String contentType) throws Exception {return minioClient.putObject(PutObjectArgs.builder().bucket(bucketName).object(fileName).stream(inputStream, fileSize, -1).contentType(contentType).build());}/*** 上传文件(如图片/PDF等)*/public ObjectWriteResponse uploadFile(File file, String bucketName, final String fileName) throws Exception {String contentType = Files.probeContentType(file.toPath());if (contentType == null) {contentType = "application/octet-stream"; // 默认类型}try (InputStream inputStream = new FileInputStream(file)) {return uploadFile(bucketName, fileName, inputStream, file.length(), contentType);}}/*** 上传本地文件*/public ObjectWriteResponse uploadLocalFile(final String bucketName, final String filePath, final String fileType) throws Exception {this.isMakeBuckets(bucketName);File file = new File(filePath);try (InputStream inputStream = new FileInputStream(file)) {return uploadFile(bucketName, file.getName(), inputStream, file.length(), fileType);}}/*** 上传MultipartFile(适用于HTTP文件上传)*/public ObjectWriteResponse uploadMultipartFile(final String bucketName, MultipartFile file, String fileName) throws Exception {this.isMakeBuckets(bucketName);final String contentType = file.getContentType();try (InputStream inputStream = file.getInputStream()) {return uploadFile(bucketName, fileName, inputStream, file.getSize(), contentType);}}/*** 上传二进制数据(如图片/PDF等) --- 会自动生成文件*/public ObjectWriteResponse uploadBytes(String bucketName, String fileName, byte[] data, final String fileType) throws Exception {this.isMakeBuckets(bucketName);try (InputStream inputStream = new ByteArrayInputStream(data)) {return uploadFile(bucketName, fileName, inputStream, data.length, fileType);}}/*** 上传文本内容 --- 会自动生成文件*/public ObjectWriteResponse uploadText(String bucketName, String fileName, String text) throws Exception {this.isMakeBuckets(bucketName);byte[] bytes = text.getBytes(StandardCharsets.UTF_8);try (InputStream inputStream = new ByteArrayInputStream(bytes)) {return uploadFile(bucketName, fileName, inputStream, bytes.length, FileType.TXT.getMimeType());}}/*** 上传JSON内容 --- 会自动生成文件*/public ObjectWriteResponse uploadJson(final String bucketName, final String fileName, final String json) throws Exception {this.isMakeBuckets(bucketName);byte[] bytes = json.getBytes(StandardCharsets.UTF_8);try (InputStream inputStream = new ByteArrayInputStream(bytes)) {return uploadFile(bucketName, fileName, inputStream, bytes.length, FileType.JSON.name());}}private void isMakeBuckets(String bucketName) throws ServerException, InsufficientDataException, ErrorResponseException, IOException, NoSuchAlgorithmException, InvalidKeyException, InvalidResponseException, XmlParserException, InternalException {boolean found = this.bucketExists(bucketName);if (!found) {this.createBuckets(bucketName);}}/*** 查看已经存在的buckets下面的文件*/public List<Result<Item>> listObject(final String bucketName) throws Exception {return listObject(bucketName, "", 1000);}public List<Result<Item>> listObject(final String bucketName, final String prefix, final int size) throws Exception {Iterable<Result<Item>> results = minioClient.listObjects(ListObjectsArgs.builder().bucket(bucketName).prefix(prefix) // 开始名称.maxKeys(size) // 最大数量.includeVersions(true).recursive(true) // 是否递归遍历子目录.build());return StreamSupport.stream(results.spliterator(), false).collect(Collectors.toList());}/*** 下载到流*/public InputStream downloadToStream(String bucketName, String fileName) throws Exception {try {return minioClient.getObject(GetObjectArgs.builder().bucket(bucketName).object(fileName).build());} catch (MinioException e) {throw new Exception("下载失败: " + e.getMessage());}}/*** 下载到本地*/public void downloadToLocal(String bucketName, String fileName, String localFilePath) throws Exception {try (InputStream inputStream = downloadToStream(bucketName, fileName)) {Path path = Path.of(localFilePath);Files.copy(inputStream, path);} catch (Exception e) {throw new Exception("文件保存失败: " + e.getMessage());}}/*** 单个文件删除*/public void deleteObject(String bucketName, String fileName) throws Exception {try {minioClient.removeObject(RemoveObjectArgs.builder().bucket(bucketName).object(fileName).build());} catch (MinioException e) {throw new Exception("删除失败: " + e.getMessage());}}/*** 批量删除(需自行遍历)*/public void batchDelete(String bucketName, List<String> objectNames) {objectNames.forEach(name -> {try {deleteObject(bucketName, name);} catch (Exception e) {// 记录日志或处理异常System.err.println("删除失败: " + name + " | 原因: " + e.getMessage());}});}/*** 获取预览地址* 有效时间默认1H** @return*/public String getPreviewFileUrl(String bucketName, String fileName) throws Exception {return getPreviewFileUrl(bucketName, fileName, 10, TimeUnit.MINUTES);}public String getPreviewFileUrl(String bucketName, String fileName, final int expiryTime, final TimeUnit timeUnit) throws Exception {return minioClient.getPresignedObjectUrl(GetPresignedObjectUrlArgs.builder().method(Method.GET).bucket(bucketName).object(fileName).expiry(expiryTime, timeUnit).build());}
}
Step7、启动类MinioApplication.java
@SpringBootApplication
@EnableConfigurationProperties(MinioProperties.class) // 显式启用
public class MinioApplication {public static void main(String[] args) {SpringApplication.run(MinioApplication.class, args);}
}
Step8、测试类---按需
import com.lting.MinioApplication;
import io.minio.ObjectWriteResponse;
import io.minio.Result;
import io.minio.messages.Bucket;
import io.minio.messages.Item;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;import java.io.File;
import java.util.List;@SpringBootTest(classes = MinioApplication.class)
public class MinioTest {@Autowiredprivate MinioUtils minioUtils;@Test@DisplayName("MinioUtils")public void properties() {System.out.println(minioUtils.getMinioProperties());}@Test@DisplayName("listBuckets")public void listBuckets() {try {List<Bucket> buckets = minioUtils.listBucket();buckets.forEach(x -> System.out.println(x.creationDate() + ", " + x.name()));} catch (Exception e) {throw new RuntimeException(e);}}@Test@DisplayName("makeBuckets")public void createBuckets() {try {minioUtils.createBuckets("test_code_create");listBuckets();} catch (Exception e) {throw new RuntimeException(e);}}@Test@DisplayName("deleteBuckets")public void deleteBuckets() {try {minioUtils.removeBucket("test_code_create");listBuckets();} catch (Exception e) {throw new RuntimeException(e);}}@Test@DisplayName("uploadLocalFile")public void uploadLocalFile() throws Exception {final String bucketName = "test_code_create";final String filePath = "C:\\Users\\xxxxx.png";final File file = new File(filePath);ObjectWriteResponse owr = minioUtils.uploadFile(file, bucketName,"2025/04/25/pig.png"); // 当在webui中创建的路径:V1/V2/V3一样的效果System.out.println(owr.etag());}@Test@DisplayName("listBuckets")public void uploadText() throws Exception {String content = "{\"url\":\"http://ip:9001/api/v1/service-account-credentials\",\"accessKey\":\"xxxx\",\"secretKey\":\"xxxxxx\",\"api\":\"s3v4\",\"path\":\"auto\"}";ObjectWriteResponse owr= minioUtils.uploadText("test_code_create", "/2025/04/25/测试.txt", content);System.out.println(owr.etag());}@Test@DisplayName("listObject")public void listObject() throws Exception {List<Result<Item>> items= minioUtils.listObject("test_code_create");items.forEach(x -> {try {Item item = x.get();System.out.println(item.etag() + "\t" + item.objectName() + "\t" + item.size() + "\t" + item.lastModified().toLocalDateTime() + "\t" + item.versionId());} catch (Exception e) {throw new RuntimeException(e);}});}@Test@DisplayName("getPreviewFileUrl")public void getPreviewFileUrl() throws Exception {System.out.println(minioUtils.getPreviewFileUrl("test_code_create", "2025/04/25/ltingzx.png"));;}@Test@DisplayName("downloadToLocal")public void downloadToLocal() throws Exception {minioUtils.downloadToLocal("test_code_create", "requirements.txt", "C:\\Users\\YiYang\\Desktop\\LLama-Factory\\req.txt");}}
关于Minio真实IP解决方法:
使用Minio的时候,生成资源的临时访问链接时,生成的地址IP是真实的IP和端口,不安全,怎么办?
minio通过“minioUtils.getPreviewFileUrl”方法生成临时链接为一般为比较长的链接,比如:“http://127.0.0.1:9001/test/xxxx_47109.png?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=gKK5g5pdjV6LWdW8XtoO%2F20250428%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20250428T100736Z&X-Amz-Expires=600&X-Amz-SignedHeaders=host&X-Amz-Signature=bd9560f5655e341263a8944545142449b1a7a393e8952eb20f9be9be9cc1391b”
---- 注意:解决方案有很多种,但是本文使用的是,短链接+Nginx代理+代理请求方案解决
Step1、前提准备
准备一个域名 比如 minio.ltingzx.com
如果没有域名,可以在本地修改hosts文件 添加: 127.0.0.1 minio.ltingzx.com
step2、修改application.yml文件
将url修改为step1中的域名;
minio:url: "http://minio.ltingzx.com" # 需要配置 指向MinIO真实地址:http://ip:9000; --- 可以使用Nginx反向代理
Step3、添加nginx.conf配置,并启动nginx
关于优化,因为我们使用的是minio存储图片等资源,所以我们可以在Nginx上面开启缓存,用以优化...;
server {listen 80;server_name minio.ltingzx.com;# 指定前端项目所在的位置location / {proxy_pass http://ip:9000; # 指向MinIO真实地址proxy_set_header Host $host;proxy_set_header X-Real-IP $remote_addr;proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;proxy_set_header X-Forwarded-Proto $scheme;}error_page 500 502 503 504 /50x.html;location = /50x.html {root html;}}
Step4、生成短链接以及访问短链接
长链接请求接口A ,接口A 生成短链接并返回;
用户拿到短链接后,请求服务的接口,服务接口拿到短链接以后,在映射关系中找到对应的长链接,直接重定向/转发/代理请求即可。
重定向:重定向是需要服务器端返回url地址,由浏览器端再次请求---此时依然会暴露真实ip和端口,且浏览器会改变地址;
转发:之所以不使用转发,核心原因是因为,转发的话需要资源和服务在同一网络---此时我们的minio在云端服务器,服务在本地,所以不适用。
代理请求:拿到短链接后,直接由服务器进行请求;然后服务器拿到的资源返回给response的outputstream流即可。
本文使用UUID临时处理,并且直接使用map方法存储:
真实开发中,我们的映射关系可以存储在数据库(比如Redis/MySQL)中,并且要设置过期时间;
使用重定向访问短链接
private final static Map<String, String> URL_MAP = new HashMap<>();
@PostMapping("/generate")public R<String> generate(@RequestParam("longUrl") String longUrl) {String shortUrl = UUID.randomUUID().toString().replace("-", "");URL_MAP.put(shortUrl, longUrl);return R.ok(shortUrl);}/*** 短链接点击跳转*重定向方法: ---- 要改变浏览器地址* 设置sendRedirect,比如response.sendRedirect(URL_MAP.get(shortURL))* 或者设置状态302,* response.setStatus(302);* response.setHeader("Location", URL_MAP.get(shortURL));* @param shortURL*/
@GetMapping("/{shortURL}")public void redirect(@PathVariable("shortURL") String shortURL, HttpServletResponse response) throws IOException, NotFoundException {System.out.println("短链接 redirect to :" + shortURL);if (URL_MAP.containsKey(shortURL)) {
// response.sendRedirect(URL_MAP.get(shortURL));response.setStatus(302);response.setHeader("Location", URL_MAP.get(shortURL));
// response.sendRedirect(URL_MAP.get(shortURL));} else {throw new NotFoundException("短链已过期");}}
服务端代理请求形式 --- 直接由服务器端发起请求
实现步骤
生成短链接:将长URL映射为短码(如Base62编码),并存储映射关系。
处理短链接请求:当访问短链接时,服务器通过短码获取长URL。
代理请求:服务器端发起HTTP请求到长URL,将响应内容返回客户端,保持地址栏不变。
@GetMapping("/{shortURL}")public void proxyRequest(@PathVariable("shortURL") String targetUrl, HttpServletResponse resp) throws IOException, NotFoundException {try (CloseableHttpClient httpClient = HttpClients.createDefault()) {HttpGet httpGet = new HttpGet(URL_MAP.get(targetUrl));try (CloseableHttpResponse response = httpClient.execute(httpGet)) {// 复制状态码resp.setStatus(response.getStatusLine().getStatusCode());// 复制响应头org.apache.http.Header[] headers = response.getAllHeaders();for (org.apache.http.Header header : headers) {resp.setHeader(header.getName(), header.getValue());}// 复制响应内容HttpEntity entity = response.getEntity();if (entity != null) {try (InputStream inputStream = entity.getContent();OutputStream outputStream = resp.getOutputStream()) {byte[] buffer = new byte[1024];int bytesRead;while ((bytesRead = inputStream.read(buffer)) != -1) {outputStream.write(buffer, 0, bytesRead);}}}}}}