京东商品图片视频批量下载与m3u8视频合并技术完整实现方案

引言

很多做京东的卖家在问：“能下载京东主图视频的软件推荐”“支持京东主图视频下载的软件有吗？”

京东商品的主图视频是混剪的重要素材，但直接下载比图片复杂。京东主图视频有两种存储格式：mp4直链和m3u8分片。m3u8格式需要下载几十个ts片段再合并，普通工具无法处理。

本文从技术角度深度解析京东商品图片视频的批量采集技术，包括主图提取、SKU图分类、m3u8视频下载合并等核心模块。类似的技术方案在一键存图中已有成熟应用。

目录

1. 京东平台技术特点分析

2. 京东图片URL原图转换

3. 京东主图提取技术

4. 京东SKU图自动分类

5. 京东视频提取与m3u8下载

6. 京东详情图提取

7. 页面加载等待策略

8. 完整采集流程实现

9. 文件存储与归档

10. 批量采集与队列管理

11. 实测数据与总结

一、京东平台技术特点分析

1.1 核心特点

京东商品页面结构相对规范，但在采集层面有其独特之处：

1.2 京东图片URL格式

京东图片URL使用n0/n1/n2标识不同尺寸：

1.3 京东视频格式

京东主图视频有两种格式：

二、京东图片URL原图转换

2.1 原图转换规则

javascript

function getJdOriginalUrl(url) { if (!url) return null; // 跳过无效图片 if (url.startsWith('data:')) return null; if (url.includes('1x1') || url.includes('blank.gif')) return null; // 去除URL参数 url = url.split('?')[0]; // n1/n2 -> n0（原图） url = url.replace(/\/n\d\//, '/n0/'); // 去除水印版本标识 url = url.replace(/\/popWaterMark\//, '/'); // 去除尺寸后缀 url = url.replace(/_\d+x\d+\./g, '.'); return url; }

2.2 转换示例

javascript

// 示例1：n1缩略图转原图 const n1Url = 'https://img13.360buyimg.com/n1/xxx.jpg'; const original = getJdOriginalUrl(n1Url); // 结果: https://img13.360buyimg.com/n0/xxx.jpg // 示例2：带水印版本转原图 const watermarkedUrl = 'https://img14.360buyimg.com/popWaterMark/xxx.jpg'; const original2 = getJdOriginalUrl(watermarkedUrl); // 结果: https://img14.360buyimg.com/xxx.jpg

三、京东主图提取技术

3.1 主图容器识别

javascript

function findJdMainContainer() { const selectors = [ '.spec-img', '.J_zoomPic', '#spec-img', '.preview-img', '.product-img' ]; for (const selector of selectors) { const element = document.querySelector(selector); if (element) return element; } return null; }

3.2 主图提取

javascript

function extractJdMainImages() { const images = []; const seen = new Set(); // 方法1：从主图容器提取 const container = findJdMainContainer(); if (container) { let url = container.src || container.getAttribute('data-lazy-img'); if (url) { url = getJdOriginalUrl(url); if (!seen.has(url)) { seen.add(url); images.push(url); } } } // 方法2：从缩略图列表提取 const thumbSelectors = [ '.spec-thumb img', '.J_thumImg', '.preview-thumb img' ]; for (const selector of thumbSelectors) { const thumbs = document.querySelectorAll(selector); for (const thumb of thumbs) { let url = thumb.src || thumb.getAttribute('data-lazy-img'); if (url) { url = getJdOriginalUrl(url); if (!seen.has(url)) { seen.add(url); images.push(url); } } } if (images.length > 0) break; } return images; }

四、京东SKU图自动分类

4.1 SKU容器识别

javascript

function findJdSkuContainer() { const selectors = [ '.sku-img-list', '.J_skuImgList', '.sku-list', '[class*="sku"]' ]; for (const selector of selectors) { const container = document.querySelector(selector); if (container && container.querySelectorAll('img').length > 0) { return container; } } return null; }

4.2 SKU图提取

javascript

function extractJdSkuImages() { const skuImages = []; const container = findJdSkuContainer(); if (!container) return skuImages; const skuItems = container.querySelectorAll('.sku-img-item, .J_skuImgItem'); for (const item of skuItems) { // 提取SKU名称（颜色/尺寸） let name = ''; const nameEl = item.querySelector('.sku-name, .J_skuName'); if (nameEl) { name = nameEl.textContent?.trim(); } if (!name) { name = item.getAttribute('title') || '规格'; } // 提取SKU图片 const img = item.querySelector('img'); if (img) { let url = img.src || img.getAttribute('data-lazy-img'); if (url) { url = getJdOriginalUrl(url); skuImages.push({ url: url, name: name }); } } } return skuImages; }

五、京东视频提取与m3u8下载

5.1 视频URL提取

javascript

function extractJdVideo() { // 方法1：从video标签提取 const videoSelectors = [ '.JDV-video video', '.video-box video', '#main-video video' ]; for (const selector of videoSelectors) { const video = document.querySelector(selector); if (video && video.src) { return { url: video.src, type: video.src.endsWith('.mp4') ? 'mp4' : 'm3u8' }; } } // 方法2：从页面数据提取 if (window.pageConfig && window.pageConfig.product) { const product = window.pageConfig.product; if (product.videoUrl) { return { url: product.videoUrl, type: product.videoUrl.endsWith('.mp4') ? 'mp4' : 'm3u8' }; } } // 方法3：从HTML提取 const html = document.documentElement.innerHTML; const match = html.match(/videoUrl["']?\s*[=:]\s*["']([^"']+\.(?:mp4|m3u8))["']/); if (match) { return { url: match[1], type: match[1].endsWith('.mp4') ? 'mp4' : 'm3u8' }; } return null; }

5.2 m3u8视频下载器

python

import os import time import requests import m3u8 from concurrent.futures import ThreadPoolExecutor class M3U8Downloader: """m3u8视频下载器，支持并行下载和自动合并""" def __init__(self, max_workers=10): self.max_workers = max_workers self.headers = { 'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36', 'Referer': 'https://item.jd.com/' } def download(self, m3u8_url, output_path): try: # 1. 解析m3u8获取ts片段列表 playlist = m3u8.load(m3u8_url, headers=self.headers) base_url = '/'.join(m3u8_url.split('/')[:-1]) + '/' segments = [] for segment in playlist.segments: if segment.uri.startswith('http'): segments.append(segment.uri) else: segments.append(base_url + segment.uri) print(f"发现 {len(segments)} 个ts片段") # 2. 创建临时目录 temp_dir = f"temp_{int(time.time())}" os.makedirs(temp_dir, exist_ok=True) # 3. 并行下载ts片段 ts_files = [] with ThreadPoolExecutor(max_workers=self.max_workers) as executor: futures = [] for i, ts_url in enumerate(segments): ts_path = os.path.join(temp_dir, f"seg_{i:05d}.ts") futures.append(executor.submit(self._download_ts, ts_url, ts_path)) ts_files.append(ts_path) for future in futures: future.result() # 4. 合并为mp4 os.makedirs(os.path.dirname(output_path), exist_ok=True) with open(output_path, 'wb') as outfile: for ts_file in ts_files: if os.path.exists(ts_file): with open(ts_file, 'rb') as infile: outfile.write(infile.read()) # 5. 清理临时文件 for ts_file in ts_files: if os.path.exists(ts_file): os.remove(ts_file) os.rmdir(temp_dir) return True except Exception as e: print(f"下载失败: {e}") return False def _download_ts(self, ts_url, ts_path, retry=3): for attempt in range(retry): try: response = requests.get(ts_url, headers=self.headers, timeout=30) if response.status_code == 200: with open(ts_path, 'wb') as f: f.write(response.content) return True except: if attempt < retry - 1: time.sleep(1) return False

六、京东详情图提取

javascript

function extractJdDetailImages() { const images = []; const seen = new Set(); const detailSelectors = [ '#detail', '.detail-content', '.J_detailContent', '.product-description' ]; for (const selector of detailSelectors) { const container = document.querySelector(selector); if (container) { const imgs = container.querySelectorAll('img'); for (const img of imgs) { let url = img.src || img.getAttribute('data-lazy-img'); if (url) { url = getJdOriginalUrl(url); if (!seen.has(url)) { seen.add(url); images.push(url); } } } if (images.length > 0) break; } } return images; }

七、页面加载等待策略

javascript

async function waitForJdPage() { // 第一重：等待DOM就绪 while (document.readyState !== 'complete') { await sleep(200); } // 第二重：等待jQuery加载（京东依赖jQuery） while (typeof jQuery === 'undefined') { await sleep(100); } // 第三重：等待图片容器 let maxWait = 30; while (maxWait-- > 0) { const container = document.querySelector('.spec-img, .J_zoomPic'); if (container) break; await sleep(500); } // 第四重：等待网络空闲 let idleCount = 0; while (idleCount < 2) { const activeRequests = performance.getEntriesByType('resource') .filter(r => r.duration === 0).length; if (activeRequests === 0) { idleCount++; } else { idleCount = 0; } await sleep(500); } // 第五重：触发懒加载 await triggerJdLazyLoad(); // 第六重：额外等待 await sleep(1000); } async function triggerJdLazyLoad() { const lazyImages = document.querySelectorAll('img[data-lazy-img]'); console.log(`发现 ${lazyImages.length} 个懒加载图片`); window.scrollTo(0, document.body.scrollHeight); await sleep(500); const steps = [0.2, 0.4, 0.6, 0.8, 1.0]; for (const step of steps) { window.scrollTo(0, document.body.scrollHeight * step); await sleep(300); } window.scrollTo(0, 0); await sleep(300); } function sleep(ms) { return new Promise(resolve => setTimeout(resolve, ms)); }

八、完整采集流程

javascript

async function collectJdProduct() { try { console.log('开始采集京东商品...'); // 1. 等待页面加载 await waitForJdPage(); // 2. 提取商品标题 const title = extractJdTitle(); console.log(`商品标题: ${title}`); // 3. 提取主图 const mainImages = extractJdMainImages(); console.log(`主图数量: ${mainImages.length}`); // 4. 提取SKU图 const skuImages = extractJdSkuImages(); console.log(`SKU图数量: ${skuImages.length}`); // 5. 提取详情图 const detailImages = extractJdDetailImages(); console.log(`详情图数量: ${detailImages.length}`); // 6. 提取视频 const video = extractJdVideo(); if (video) { console.log(`视频类型: ${video.type}`); } return { success: true, title: title, mainImages: mainImages, skuImages: skuImages, detailImages: detailImages, video: video }; } catch (error) { console.error(`采集失败: ${error.message}`); return { success: false, error: error.message }; } } function extractJdTitle() { const selectors = ['.sku-name', '.product-title', 'h1']; for (const selector of selectors) { const el = document.querySelector(selector); if (el && el.textContent) { const title = el.textContent.trim(); if (title.length > 5) return title; } } return document.title || '京东商品'; }

九、文件存储与归档

javascript

class StorageManager { constructor(basePath = './downloads/jd') { this.basePath = basePath; } saveProduct(productData) { const safeTitle = this.sanitizeFilename(productData.title); const productDir = `${this.basePath}/${safeTitle}`; // 创建目录结构 const dirs = ['视频', '主图', 'SKU图', '详情图']; for (const dir of dirs) { this.ensureDir(`${productDir}/${dir}`); } const result = { main: [], sku: [], detail: [], video: null }; // 保存主图 productData.mainImages.forEach((url, idx) => { const path = `${productDir}/主图/主图_${idx + 1}.jpg`; result.main.push({ url, path }); }); // 保存SKU图 productData.skuImages.forEach(sku => { const safeName = this.sanitizeFilename(sku.name); const path = `${productDir}/SKU图/${safeName}.jpg`; result.sku.push({ url: sku.url, path, name: sku.name }); }); // 保存详情图 productData.detailImages.forEach((url, idx) => { const path = `${productDir}/详情图/详情图_${idx + 1}.jpg`; result.detail.push({ url, path }); }); // 保存视频 if (productData.video) { const path = `${productDir}/视频/视频.mp4`; result.video = { url: productData.video.url, path }; } return result; } sanitizeFilename(name) { return name.replace(/[\\/*?:"<>|]/g, '_').substring(0, 200); } ensureDir(path) {} }

十、批量采集与队列管理

javascript

class BatchCollector { constructor(concurrency = 1) { this.concurrency = concurrency; this.queue = []; this.running = 0; this.results = []; } async collectAll(urls) { const results = []; for (const url of urls) { const result = await this.collectOne(url); results.push(result); } return results; } async collectOne(url) { try { const result = await collectJdProduct(); const storage = new StorageManager(); const saved = storage.saveProduct(result); return { success: true, url, data: saved }; } catch (error) { return { success: false, url, error: error.message }; } } }

十一、实测数据与总结

11.1 性能数据

11.2 各类型素材采集结果

11.3 总结

京东商品图片视频批量采集的核心技术点：

1. 原图转换：将n1/n2替换为n0，获取最大分辨率原图

2. SKU图分类：从SKU容器中提取属性名称并关联图片

3. m3u8视频处理：解析m3u8索引，下载ts片段并合并为mp4

4. 懒加载处理：触发data-lazy-img属性的图片加载

类似一键存图的工具已经将这些技术封装成成熟产品，用户无需编写代码，只需复制商品链接即可自动完成京东商品素材的采集，m3u8视频自动合并，SKU图自动按颜色/尺寸分类，将原来5-10分钟的手工整理压缩到30秒。

免责声明：本文内容仅供技术交流和学习参考。电商平台的数据采集行为可能涉及平台服务条款、著作权法等法律问题。请确保遵守目标网站的《用户协议》和相关法律法规。因不当使用引发的法律风险由使用者自行承担。