Source: https://patents.google.com/patent/US10031893B2/en
Timestamp: 2019-11-15 06:14:33
Document Index: 521123645

Matched Legal Cases: ['Application No. 201380030239', 'Application No. 201380038361', 'Application No. 201380038452', 'Application No. 201380038361', 'Application No. 201380038452', 'Application No. 201380030239', 'application No. 13728612', 'Application No. 13745236', 'application No. 13745925', 'application No. 102118689']

US10031893B2 - Transforming data to create layouts - Google Patents
US10031893B2
US10031893B2 US15/416,953 US201715416953A US10031893B2 US 10031893 B2 US10031893 B2 US 10031893B2 US 201715416953 A US201715416953 A US 201715416953A US 10031893 B2 US10031893 B2 US 10031893B2
US15/416,953
US20170139877A1 (en
2017-01-26 Application filed by Microsoft Technology Licensing LLC filed Critical Microsoft Technology Licensing LLC
2017-01-26 Priority to US15/416,953 priority patent/US10031893B2/en
2017-01-31 Assigned to MICROSOFT CORPORATION reassignment MICROSOFT CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: FREIER, Nathan George, LEE, DAVID BENJAMIN, TUMANOV, ILYA
2017-05-18 Publication of US20170139877A1 publication Critical patent/US20170139877A1/en
2018-07-24 Publication of US10031893B2 publication Critical patent/US10031893B2/en
This patent application is a continuation application of, and claims priority to, co-pending, commonly-owned U.S. patent application Ser. No. 13/552,416, entitled “Transforming Data to Create Layouts,” filed Jul. 18, 2012, which is incorporated herein by reference in its entirety.
1. A computer-implemented method for transforming data to generate a layout, the computer-implemented method comprising:
obtaining data comprising an image and a content item;
analyzing the image to identify:
an invariant region of the image, the invariant region comprising a region of the image wherein a variation of at least one feature is below a threshold to achieve a contrast between the content item and the image;
selecting a design of the plurality of designs to generate the layout, the design comprising a viewport frame, a salient region placement frame, and a content placement frame, and the selecting being based at least in part on:
determining that the invariant region corresponds to the content placement frame; and
generating the layout based at least in part on the design.
2. The method of claim 1, further comprising, prior to determining that the salient region corresponds to the salient region placement frame, performing at least one manipulation to the image.
3. The method of claim 2, wherein the at least one manipulation comprises zooming, panning, scrolling, cropping, extending, infilling, or seam carving of the image.
4. The method of claim 1, wherein selecting the design comprises:
5. The method of claim 1, wherein the viewport frame defines viewable limits of the layout.
6. The method of claim 1, wherein the at least one feature comprises color, luminance, sharpness, blur, or pattern.
7. The method of claim 1, wherein generating the layout comprises:
placing the content item such that at least a second portion of the content item is located in a second position corresponding to the content placement frame.
8. The method of claim 1, wherein analyzing the image to identify the salient region is based at least in part on one of:
analyzing color differences at or within the salient region;
performing facial recognition processing on the image; or
analyzing blurring or sharpness at or within the salient region.
obtain data comprising visual content and a content item;
an invariant region of the image, the invariant region comprising a region of the image wherein a variation of a feature is below a threshold to achieve a contrast between the content item and the image;
select, based at least partially upon the comparison of the data, a design of the plurality of designs, the design defining a layout and including a viewport frame, a salient region placement frame, and a content placement frame, and the selecting being based at least in part on:
generate a layout based at least in part on the design.
10. The computer storage medium of claim 9, wherein the feature includes at least one of color, luminance, sharpness, blur, or pattern.
11. The computer storage medium of claim 9, wherein the computer readable instructions further cause the computer to:
adjust a position of the image relative to the viewport frame to locate the salient region at a first location within the layout corresponding to the salient region placement frame; and
overlay the content item at a location within the layout that corresponds to a second location of the content placement frame within the design.
12. The computer storage medium of claim 11, wherein the computer readable instructions further cause the computer to:
compare the invariant region to second locations respectively corresponding to content placement frames in the individual designs of the plurality of designs;
13. The computer storage medium of claim 9, wherein the computer readable instructions further cause the computer to:
select the design based at least in part on the scaled version of the image causing the salient region to correspond to the salient region placement frame and the invariant region to correspond to the content placement frame.
14. The computer storage medium of claim 9, wherein analyzing the image to identify the salient region is based on at least one of:
analyzing color distributions at or within the salient region;
performing object recognition; or
performing facial recognition.
15. The method of claim 9, wherein the invariant region excludes the salient region.
analyze the visual content to identify a salient region of the visual content and an invariant region of the visual content, the salient region comprising a subject of the visual content and the invariant portion comprising a first portion of the visual content wherein variation within the visual content is below a threshold;
select, based at least partially upon the comparison of the data, a design defining a layout, the design comprising a viewport frame defining visible limits of the layout, a salient region placement frame corresponding to a first location at which the salient region of the visual content is to be placed, and a content placement frame corresponding to a second location at which the content item is to be placed within the layout;
load the data into the design based, at least partially, upon the salient region, the invariant region, the salient region placement frame, and the content placement frame; and
generate the layout, the layout comprising at least the subject of the visual content and the content item.
based at least in part on comparing the data to the plurality of designs, determine a score for each design of the plurality of designs, the score representative of a degree to which each design of the plurality of designs relates to the data;
18. The apparatus of claim 16, wherein analyzing the visual content to identify the salient region of the visual content is based at least in part on analyzing brightness data at or within the salient region.
19. The apparatus of claim 16, wherein the application program further causes the processor to determine, based at least in part the comparison of the data, that the design matches the data based at least in part on:
determining that at least a third portion of the invariant portion fits within the content placement frame.
20. The apparatus of claim 16, wherein the content item comprises text or another element of content.
US15/416,953 2012-07-18 2017-01-26 Transforming data to create layouts Active US10031893B2 (en)
US13/552,416 Continuation US9595298B2 (en) 2012-07-18 2012-07-18 Transforming data to create layouts
US16/017,982 Continuation US20180300293A1 (en) 2012-07-18 2018-06-25 Transforming data to create layouts
US20170139877A1 US20170139877A1 (en) 2017-05-18
US10031893B2 true US10031893B2 (en) 2018-07-24
CN1770174A (en) 2004-10-20 2006-05-10 微软公司 Parsing hierarchical lists and outlines
CN102362368A (en) 2009-03-03 2012-02-22 富士胶片株式会社 Method for producing light-emitting display device, light-emitting display device and light-emitting display
TW201213768A (en) 2010-06-18 2012-04-01 Nokia Corp Method and apparatus for providing smart zooming of a geographic representation
US20140026038A1 (en) 2012-07-18 2014-01-23 Microsoft Corporation Transforming data to create layouts
CN102360368B (en) 2011-10-09 2014-07-02 山东大学 Web data extraction method based on visual customization of extraction template
"Captivate 4: Produce Interactive Simulations and Demonstrations", Retrieved on Dec. 27, 2011 at: «http://cit.jmu.edu/useruploads/files/captivate4tutorial.pdf«, Center for Instructional Technology, Carrier Library & East Campus Library, 11 pages.
"Feedback", Retrieved Feb. 5, 2014, at: «http://www.dictionary.reference.com/browse/feedback» Dictionary.com, 3 pages.
"Flexible Authoring Solutions for Learning Professionals", Retrieved at «http://www.adobe.com/resources/elearning/pdfs/flexible_authoring_solutions_wp.pdf», Adobe Systems, Inc., 2007, 4 pages.
"Fonts", Dec. 5, 2016, «https://msdn.microsoft.com/en-us/library/windows/desktop/dn742483(v=vs.85).aspx, pp. 1-pp. 7.
"How to Configure Desktop Themes in Windows 2000", Retrieved on Sep. 26, 2013 at : «http://support.microsoft.com/kb/2578741», 2007, 2 pages.
"Maximum Volume Sizes", retrieved on Sep. 26, 2013 at «https://technet.microsoft.com/en-us/library/cc938432.aspx», Microsoft, 2013, 3 pages.
"NTFS.com: Complete info source: NTFS & FAT file systems and data recovery" retrieved on Sep. 26, 2013, at «http://ntfs.com/ntfs.htm», 4 pages.
"Present" | Define Present at Dictionary.com, retrieved on Apr. 3, 2015 at «http://dictionary.reference.com/browse/present?s=t», Dictionary.com, 2015, 9 pages.
"Script and Font Support in Windows", Dec. 5, 2016, «https://msdn.microsoft.com/ globalization/mt791278», Developer Network, 11 Pages.
"Suggest" | Define Suggest at Dictionary.com, retrieved on Apr. 3, 2015 at «http://dictionary.reference.com/browse/suggest?s=t», Dictionary.com, 2015, 5 pages.
Abidi, "Intelligent Information Personalization: From Issues to Strategies", In Panagiotis Germanakos (EDS) Intelligent User Interfaces: Adaptation and Personalization Systems and Technologies, IGI Global Press, 2008, 21 pages.
Beale, "Supporting serendipity: using ambient intelligence to augment user exploration for data mining and web browsing", International Journal of Human-Computer Studies, vol. 65, No. 5, 2007, 26 pages.
Blokzijl, et al., "The Effect of Text Slides Compared to Visualizations on Learning and Appreciation in Lectures", Retrieved at «http://ieeexplore.ieee.org/xpl/freeabs_all.jsp?arnumber=4464074» IEEE International Professional Communication Conference, 2007, IPCC2007 2007 IEEE Oct. 1-3, 2007, 9 pages.
Bradley, "Last Patch Tuesday for Windows 2000 and Windows XP SP2", Dec. 6, 2016, «http://www.pcworld.com/article/2007 5 2/last_patch_ tuesday for_ windows_2000_and_windows_xp_sp2.html», Jul. 9, 2010, pp. # 1-pp. # 2, 2 pages.
Bucaro, "Configure Folder Options", Retrieved on Sep. 26, 2013 at «http://bucarotechelp.com/computers/wintips/94052503.asp», Bucaro TecHelp, 6 pages.
Chinese Office Action dated Apr. 13, 2016 for Chinese Patent Application No. 201380030239.7, a counterpart foreign application of U.S. Appl. No. 13/491,612, 16 pages.
Chinese Office Action dated Jul. 1, 2016 for Chinese Patent Application No. 201380038361.9, a counterpart foreign application of U.S. Appl. No. 13/552,662, 22 pages.
Chinese Office Action dated Jul. 4, 2016 for Chinese Patent Application No. 201380038452.2, a counterpart foreign application of U.S. Appl. No. 13/551,638, 19 pages.
Chinese Office Action dated Mar. 2, 2017 for Chinese Patent Application No. 201380038361.9, a counterpart foreign application of U.S. Appl. No. 13/552,662, 13 pages.
Chinese Office Action dated Mar. 6, 2017 for Chinese Patent Application No. 201380038452.2, a counterpart foreign application of U.S. Appl. No. 13/551,638, 16 pages.
Chinese Office Action dated Nov. 10, 2016 for Chinese Patent Application No. 201380030239.7, a counterpart foreign application of U.S. Appl. No. 13/491,612, 12 pages.
Choi, et al., "Exploring Factors of Media Characteristic Influencing Flow in Learning through Virtual Worlds", Computers & Education, vol. 57, No. 4, 2011, 24 pages.
Dobrikov, et al, "The Architecture of a Corporate Information and News Engine", International Conference on Computer Systems and Technologies, ComSysTech, '2003, 6 pages.
El-Bakry, "Face Detection Using Neural Networks and Image Decomposition" In Proceedings of the 2002 International Joint Conference on Neural Networks IEEE, vol. 1, 2002, 6 pages.
Esposito, "More Windows 2000 UI Goodies: Extending Explorer Views by Customizing Hypertext Template Files", Available at «https://msdn.microsoft.com/en-us/magazine/cc302292.aspx» MSDN Magazine, 2000, 12 pages.
European Office Action dated Jul. 22, 2016 for European patent application No. 13728612.6, a counterpart foreign application of U.S. Appl. No. 13/491,612, 8 pages.
European Office Action dated Jul. 22, 2016 for European Patent Application No. 13745236.3, a counterpart foreign application of U.S. Appl. 13/552,662, 9 pages.
European Office Action dated Jul. 25, 2016 for European patent application No. 13745925.1, a counterpart foreign application of U.S. Appl. No. 13/551,638, 9 pages.
Feng, et al, "Salient Object Detection by Composition" IEEE International Conference on Computer Vision (ICCV), 2011, 8 pages.
Griebel, "Repository Support for Visualization in Relational Databases", CiteSeerX, available at: «http://www.citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.12.2225»1996, 121 pages.
Hinz, et al, "Personalization-Based Optimization of Web Interfaces for Mobile Devices", Mobile Human-Computer Interaction-MobileHCI2004, vol. 3160, 2004, 12 pages.
Hinz, et al, "Personalization-Based Optimization of Web Interfaces for Mobile Devices", Mobile Human-Computer Interaction—MobileHCI2004, vol. 3160, 2004, 12 pages.
Huang, et al, "Header Metadata Extraction from Semi-structured Documents Using Template Matching*", Lecture Notes in Computer Science, vol. 4278, 2006, 10 pages.
International Search Report dated Jan. 29, 2014 in International Application No. PCT/US13/050961, 15 pages.
Jackson, "Windows 2000 Root Kit Analysis: Attack of Nov. 1, 2003", Gatech, available at: «http://www.users.ece.gatech.edu/owen/Research/HoneyNet/Quarterly/Analysis_of_Windows_2000_root-kit_htm»Sep. 26, 2013, 4 pages.
Jovanovic, et al, "Automating Semantic Annotation to Enable Learning Content Adaptation", Lecture Notes in Computer Science, vol. 4018, 2006, 10 pages.
Jovanovic, et al, "Ontology-based Automatic Annotation of Learning Content" International Journal on Semantic Web and Information Systems (IJSWIS), vol. 2, No. 2, 2006, 29 pages.
Kaiser, et al, "Information Extraction, A Survey", Media Processing, Vienna University of Technology, 2005, 32 pages.
Kamba, et al, "The Krakatoa chronicle an interactive, personalized, newspaper on the web", In Proceedings of the Fourth International World Wide Web Conference, 1995, 15 pages.
Kapp, "Virtual Worlds and Compliance", retrieved on Dec. 23, 2011, at «http://www.karlkapp.com/virtual-worlds-and-compliance/», Kapp Notes, 2011, 5 pages.
Kuan, et al, "Color-Based Image Salient Region Segmentation Using Novel Region Merging Strategy", IEEE Transactions on Multimedia, vol. 10, No. 5, 2008, 14 pages.
Linden, "This Project Viewer Lets Landowners Control Environment Settings" retrieved on Dec. 23, 2011, at «http://www.community.secondlife.com/t5/LandiThis-Project-Viewer-Lets-Landowners-Control-Environment-Settings/ba-p/925883«, Second Life, 2011, 3 pages.
Mansmann, et al, "Exploring OLAP Aggregates with Hierarchical Visualization Techniques", Applied computing 2007: Proceedings of the 2007 ACM symposium on Applied computing, 2007, 7 pages.
Mikolajczyk, et al, "A Performance Evaluation of Local Descriptors", IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, No. 10, 2005, 16 pages.
Office action for U.S. Appl. No. 13/491,612, dated Jul. 16, 2014, Michelstein et al, "Transforming Data into Consumable Content", 7 pages.
Office action for U.S. Appl. No. 13/491,612, dated Jul. 16, 2014, Michelstein, et al., "Transforming Data into Consumable Content", 7 pages.
Office action for U.S. Appl. No. 13/551,638, dated Apr. 9, 2015, Lee et al., "Abstract Relational Model For Transforming Data Into Consumable Content", 26 pages.
Office action for U.S. Appl. No. 13/551,638, dated Dec. 12, 2016, Lee et al., "Abstract Relational Model For Transforming Data Into Consumable Content", 31 pages.
Office Action for U.S. Appl. No. 13/551,638, dated Feb. 10, 2014, Lee, "Transforming Data into Consumable Content", 32 pages.
Office action for U.S. Appl. No. 13/551,638, dated Jun. 26, 2014, Lee et al, "Abstract Relational Model for Transforming Data Into Consumable Content" , 29 pages
Office Action for U.S. Appl. No. 13/551,638, dated Nov. 6, 2014, Lee et al, "Abstract Relational Model for Transforming Data Into Consumable Content", 22 pages.
Office action for U.S. Appl. No. 13/551,638, dated Oct. 23, 2015, Lee et al., "Abstract Relational Model For Transforming Data Into Consumable Content", 22 pages.
Office Action for U.S. Appl. No. 13/551,638, dated Oct. 28, 2013, Lee, "Transforming Data into Consumable Content", 19 pages.
Office action for U.S. Appl. No. 13/552,416, dated Dec. 18, 2015, Lee et al., "Transforming Data to Create Layouts", 13 pages.
Office action for U.S. Appl. No. 13/552,416, dated Jan. 14, 2015, Lee et al, "Transforming Data to Create Layouts", 12 pages.
Office action for U.S. Appl. No. 13/552,416, dated Jul. 2, 2015, Lee et al., "Transforming Data to Create Layouts", 13 pages.
Office action for U.S. Appl. No. 13/552,416, dated Jul. 7, 2016, Lee et al., "Transforming Data to Create Layouts", 14 pages.
Office Action for U.S. Appl. No. 13/552,662, dated Jul. 16, 2014, Michelstein et al, "Creating Variations When Transforming Data Into Consumable Content", 8 pages.
Office Action for U.S. Appl. No. 13/552,662, dated Jul. 16, 2014, Michelstein et al., "Creating Variations When Transforming Data Into Consumable Content", 7 pages.
Office action for U.S. Appl. No. 13/552,662, dated Nov. 6, 2014, Michelstein et al, "Creating Variations When Transforming Data Into Consumable Content", 5 pages.
Office action for U.S. Appl. No. 14/640,446, dated May 7, 2015, Michelstein et al., "Transforming Data into Consumable Content", 5 pages.
Office action for U.S. Appl. No. 14/670,476 dated Sep. 15, 2015, Michaelstein et al., "Creating Variations when Transforming Data into Consumable Content", 7 pages.
Office action for U.S. Appl. No. 14/670,476, dated Jan. 29, 2016, Michelstein et al., "Creating Variations when Transforming Data into Consumable Content", 7 pages.
Office action for U.S. Appl. No. 14/670,476, dated May 12, 2016, Michelstein et al., "Creating Variations when Transforming Data into Consumable Content", 9 pages.
Office action for U.S. Appl. No. 14/670,476, dated Sep. 9, 2016, Michelstein et al., "Creating Variations when Transforming Data into Consumable Content", 14 pages.
Office action for U.S. Appl. No. 14/954,447, dated Jan. 5, 2016, Michaelstein et al., "Transforming Data into Consumable Content", 5 pages.
Office action for U.S. Appl. No. 14/954,447, dated Jul. 14, 2016, Michaelstein et al., "Transforming Data into Consumable Content", 10 pages.
Office action for U.S. Appl. No. 14/954,447, dated Nov. 3, 2016, Michaelstein et al., "Transforming Data into Consumable Content", 12 pages.
PCT International Search Report and Written Opinion for Application No. PCT/US2013/042954, dated Oct. 4, 2013, 20 pages.
PCT International Search Report and Written Opinion for Application No. PCT/US2013/050964, dated Nov. 12, 2013, 18 pages.
PCT International Search Report and Written Opinion for Application No. PCT/US2013/050968, dated Jan. 27, 2014, 16 pages.
PNNL: "Cognitive Informatics"-Interactive e-Leaming Approach, Retrieved on Dec. 27, 2011 at: «http://www.onl.gov/coginformatics/elearning_approach.stm» Pacific Northwest National Laboratory, Operated by Batelle for the US Dept. of Energy, 2008, 2 pages.
PNNL: "Cognitive Informatics"—Interactive e-Leaming Approach, Retrieved on Dec. 27, 2011 at: «http://www.onl.gov/coginformatics/elearning_approach.stm» Pacific Northwest National Laboratory, Operated by Batelle for the US Dept. of Energy, 2008, 2 pages.
Rowe, et al, "Implementation and Evaluation of an Adaptive Multimedia Presentation System (AMPS) with Contextual Supplemental Support Media" Advances in Multimedia (MMEDIA), 2010 Second International Conference on Advances in Multimedia, 2010, 6 pages.
Savill, "How can I modify the icon spacing on the desktop?", Dec. 5, 2016, « http://windowsitpro.com/systems-management/how-can-i-modify-icon-spacing-desktop», Sep. 27, 2001, 2 pages.
Schrier, et al, "Adaptive Layout for Dynamically Aggregated Documents" Proceedings of the 13th International Conference on Intelligent User Interfaces, 2008, 10 pages.
Schultz, "Associating File Types with Multiple Applications" at: «https://msdn.microsoft.com/en-us/library/ms977154.aspx», Microsoft Developer Network, 2001, 5 pages.
Snakefoot, "Disable visual effects to make the desktop faster", Dec. 5, 2016, «http://smallvoid.com/article/windows-visual-effects.html», Smallvoid.com, pp. # 1-pp. # 2.
Stewart, et al, "Software Environments for Scenario-based Learning" at «http://www.akoaotearoa.ac.nz/download/ng/file/group-6/n3498-software-environments-for-scenario-based-learning-a-snapshot-comparison-of-some-available-tools.pdf», 2009, 24 pages.
Stolte, et al, "Multiscale Visualization Using Data Cubes" IEEE Transactions on Visualization and Computer Graphics, Vo. 9, No. 2, 2003, 12 pages.
Taiwanese Office Action dated Aug. 29, 2016 for Taiwanese patent applicatoin No. 102118689, a counterpart foreign application of U.S. Pat. No. 8,990,140.
Taiwanese Office Action dated Dec. 27, 2016 for Taiwanese patent application No. 102118689, a counterpart foreign application of U.S. Pat. No. 8,990,140.
Thurrott, "Microsoft Ships Final Windows 2000 Update", Dec. 6, 2016, «http://windowsitpro.com/windows-client/microsoft-ships-final-windows-2000-update», pp. 1-pp. 5.
U.S. Notice of Allowance dated May 18, 2014, in U.S. Appl. No. 13/491,612.
Wichary, "Mouse 2006", retrieved Sep. 26, 2013, at «http://www.guidebookgallery.org/screenshots/mouse», Graphical User Interface Gallery Guidebook, 2002, 26 pages.
Witten, et al, "Importing Documents and Metadata into Digital Libraries: Requirements Analysis and Extensible Architecture", Lecture Notes in Computer Science, vol. 2458, 2002, 10 pages.
Wunsche, "A Survey, Classification and Analysis of Perceptual Concepts and their Application for the Effective Visualisation of Complex Information", Proceeding APVis '04 Proceedings of the 2004 Australasian symposium on Information Visualisation, vol. 35, 2004, 8 pages.
Xiao et al., "Automatic semantic annotation based on ontology and knowledge based," May 2011, Application Research of Computers, available at: http://en.cnki.com.cn/Article_en/CJFDTOTAL-JSYJ201105043.htm, 2 pages.
Yang, et al, "Detecting Faces in Images: A Survey", In IEEE Tansactions on Pattern Analysis and Machine Intelligence, vol. 24, No. 1, Jan. 2002, 25 pages.
US9595298B2 (en) 2017-03-14
JP6215328B2 (en) 2017-10-18 Reading mode for interactive slide presentation with attached notes
US9595298B2 (en) 2017-03-14 Transforming data to create layouts
WO2015043525A1 (en) 2015-04-02 Method, apparatus, and system for picture sharing
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:LEE, DAVID BENJAMIN;FREIER, NATHAN GEORGE;TUMANOV, ILYA;REEL/FRAME:041138/0719
Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNOR:MICROSOFT CORPORATION;REEL/FRAME:042124/0191