Neural Networks and Chainmaille (Project Planning)
View previous topic | View next topic >
Post new topic Reply to topic
M.A.I.L. Forum Index -> Knitting Circle
   
Author Message

Joined: August 05, 2010
Posts: 623
Submissions: 28
Location: Bar Harbor, ME, USA

Neural Networks and Chainmaille (Project Planning)
Reply with quote
Posted on Sun Jan 07, 2018 3:53 pm
Link to Post: Link to Post

Greetings reader,

I've been bouncing an idea around in my head for the past 4ish months and feel like I should get it a bit more out in the open for feedback. Maybe this will motivate me to actually sit down and put some effort into it.

For those not heavily integrated into tech as I am, the possibilities of what computers can do has drastically changed in the past couple years. There are these things called neural networks which are really starting to do impressive things that were once thought to be extremely difficult. Think the voice recognition stuff on your phone, google search results. Or maybe you don't have that stuff. How about (for lorraine) this link where you draw edges and it textures it to be a cat. They even have 3d-from-2d face rendering over here.

So my thoughts have been tinkering around with interesting (fun and maybe useful) things to do with chainmaille. The big one that I've been juggling around for awhile is a 2d to 3d representation of weaves/projects. In fact, it could potentially do lots of crazy stuff like make guesses on the material used, ARs of the rings, and possibly even the actual weave used. (Feel free to suggest more possible things and I'll provide feedback.)

But let's not get ahead of ourselves. This isn't going to be done in a weekend or anything (estimated year+?). I do have the hardware to train these sorts of networks, but there is quite a bit of upfront work that needs to be done prior to actually training. Here's my current specific details of what I've been thinking (will be updated based on the discussions).


1. Creation of dataset 1 -- Learning to find rings
The plan is to create a fully synthetic training set. I'm probably going to learn blender's scripting syntax to automate the creation, because neural networks are notorious to heavily rely on the training dataset being large (the big competitions typically have upwards of 5 million unique images to train on). They're doing broad stuff, but I think this dataset would need to be on the order of about 10,000. There are 2 big reasons for using a synthetic dataset: 1. Can increase size without much effort (just compute time). 2. Labels (correct answer for where each ring is located in the image) are known.

2. Sub-Network 1 -- Ring Instance Predictor
Predict where rings are located (region proposal) and which pixels are specifically that ring (instance segmentation).

3. Sub-Network 2 -- Ring Shape Prediction
Predict various features about an individual ring. These items include: AR, WD (in pixels), 2d center location (3d next step), 3d orientation (yaw, pitch, roll), material.

4. Sub-Network 3 --Weave Assembler
Assign 3d position for each instance to merge all identified rings into a 3d model of a weave.

5. Creation of dataset 2 -- Classification of Weaves
Again, a fully synthetic dataset would be ideal. However, this might be a bit more difficult. Typically, around 1000ish of each class would be the minimum that people attempt to train on. Here we run into the issue of not having clear mathematical representations for the vast majority of weaves as well as lots of weaves being very close to each other (visually and interaction-wise).

6. Sub-Network 4 -- Classifying Images into Existing Weaves
I don't currently have high hopes for this, given the similarities between weaves. But hey, might as well include it in the lists. Another idea, although not as impressive, is to classify by MAIL tags.

7. Build applications for others to use.
I'm most familiar with tensorflow as my neural network software. Conveniently, they support smartphones (both android and ios. I don't have any apple products, so I won't be able to build anything for iphones (literally need a mac machine to compile for mac). I do have a big interest to build it up for android, though.


while(!project.isFinished())
project.addRing();
// Maille Code V2.0 T7.1 R5.6 Eo.n Fper MFe.s Wsm Caws G0.8-1.6 I2.4-8.0 Pn Dcdejst Xw1 S07

Joined: June 15, 2012
Posts: 19
Submissions: 0
Location: Felton, California, USA

Reply with quote
Posted on Fri Jan 12, 2018 7:11 pm
Link to Post: Link to Post

I have some experience with neural nets, but it's all pretty old, from the bad old days when it was just getting started and a training set was considered astoundingly rich with 1000 cases or so, so I don't know that I can be of much help with this project.

I do agree with the notion of starting with synthetic training data though. Otherwise it would be far too tedious to specify "truth".

Joined: August 05, 2010
Posts: 623
Submissions: 28
Location: Bar Harbor, ME, USA

Reply with quote
Posted on Tue Jan 16, 2018 11:49 am
Link to Post: Link to Post

So I was searching around to get started and I found this nifty old article over here: How to Make Chainmaille in Blender. I initially was just going to make physically infeasible scenes, but that article goes into adding physics simulations into blender.

I'll still need to learn a bit of the Blender-based python scripting, but it seems like I have a pretty good starting point now.

Also, is anyone aware of a way to do a least-squares fit of a point-cloud to a toroid? I'm very familiar with doing things like ellipse-fits with all points visible, but aren't quite sure what to do with only a fraction of the entire surface visible. I think it might be better to predict a point-cloud of an instanced ring (and then minimize the toroid fit) than have the network predict the yaw/pitch/roll.


while(!project.isFinished())
project.addRing();
// Maille Code V2.0 T7.1 R5.6 Eo.n Fper MFe.s Wsm Caws G0.8-1.6 I2.4-8.0 Pn Dcdejst Xw1 S07

Joined: August 05, 2010
Posts: 623
Submissions: 28
Location: Bar Harbor, ME, USA

Reply with quote
Posted on Sat Jan 20, 2018 4:59 pm
Link to Post: Link to Post

As per this comment...

Karpeth wrote:
Inlevelse your tool idea, but I feel you need to and tags or something like it for it to function? Am I off, or How would you add the information?


Sub-network 4 (item 6) could classify weaves/tags. The training dataset could hypothetically be the current state of the weave library (or even the gallery).

In addition to that, sub-network 3 (the weave-assembler) would create a 3D model. Given a 3D model, one could use tools like a physics collision engine to optimize theoretical minimum ring AR sizes, rings per unit length, flexibility, tessellation pattern, and a whole bunch of other stuff. In fact, this really old discussion touched on these algorithmic ideas (in an attempt to search the "possible weave" space).


while(!project.isFinished())
project.addRing();
// Maille Code V2.0 T7.1 R5.6 Eo.n Fper MFe.s Wsm Caws G0.8-1.6 I2.4-8.0 Pn Dcdejst Xw1 S07

Joined: August 30, 2010
Posts: 774
Submissions: 33

Reply with quote
Posted on Sat Jan 20, 2018 5:40 pm
Link to Post: Link to Post

I would love to see a program capable of generering new weaves.

Ten years ago, I would have Said it would be impossible.


Total Nerd: MScDS, Mailler, Gamer.

Joined: April 30, 2018
Posts: 63
Submissions: 9
Location: San Diego, CA

Reply with quote
Posted on Sat May 26, 2018 7:34 pm
Link to Post: Link to Post

Zombie thread!

Just found this and I'd be interested in trying out the weaves that the AI comes up with. It's very similar to the "SkyKnit" AI knitting and crochet projects on Ravelry, which looked like they were a lot of fun and also were quite the challenge for people to try to make some actual physical objects out of the patterns the AI was generating.

Joined: August 05, 2010
Posts: 623
Submissions: 28
Location: Bar Harbor, ME, USA

Reply with quote
Posted on Tue May 29, 2018 10:27 am
Link to Post: Link to Post

Don't worry. This is not dead yet. A bunch new generative adversarial network designs are coming up in the literature to actually do neat new creative works. Haven't seen one that enforces physical concurrence yet, though.


while(!project.isFinished())
project.addRing();
// Maille Code V2.0 T7.1 R5.6 Eo.n Fper MFe.s Wsm Caws G0.8-1.6 I2.4-8.0 Pn Dcdejst Xw1 S07

Joined: August 05, 2010
Posts: 623
Submissions: 28
Location: Bar Harbor, ME, USA

Reply with quote
Posted on Sat Nov 23, 2019 5:01 pm
Link to Post: Link to Post

If there were any lurkers that remember this idea...

I was recently perusing features I hadn't looked at in tensorflow...
I don't know how good their "geometry" model is, but the fact that there is one suggests that there could be an auto-encoder approach might work. The area is definitely in its infancy, but I might start tinkering with it to see if it can detect individual rings in a scene to measure things like AR. (Wouldn't that be a fun app to have on your phone instead of having to break out the measurement tools?)
Going all the way to finding patterns in weaves might be a bit more difficult (mostly due to high occlusion, un-enforced physical restrictions).


while(!project.isFinished())
project.addRing();
// Maille Code V2.0 T7.1 R5.6 Eo.n Fper MFe.s Wsm Caws G0.8-1.6 I2.4-8.0 Pn Dcdejst Xw1 S07

Joined: August 30, 2010
Posts: 774
Submissions: 33

Reply with quote
Posted on Mon Jan 13, 2020 9:56 pm
Link to Post: Link to Post

Have you had any further luck?


Total Nerd: MScDS, Mailler, Gamer.

Joined: August 05, 2010
Posts: 623
Submissions: 28
Location: Bar Harbor, ME, USA

Reply with quote
Posted on Mon Jan 13, 2020 11:23 pm
Link to Post: Link to Post

I'd be lying if I said I've found time to sit down and put forth substantial effort into this.

I did dig a bit deeper into the geometry library, and it seems to be rather bare-bones by only including a mesh-import (and just about no documentation). I'm waiting for those tools to mature a bit more. As it stands learning blender and generating images is still a better option. Maybe a couple more months before I breakdown to learn blender enough to generate images.


while(!project.isFinished())
project.addRing();
// Maille Code V2.0 T7.1 R5.6 Eo.n Fper MFe.s Wsm Caws G0.8-1.6 I2.4-8.0 Pn Dcdejst Xw1 S07

Joined: February 13, 2020
Posts: 46
Submissions: 0
Location: USA

Reply with quote
Posted on Mon Feb 17, 2020 1:32 am
Link to Post: Link to Post

I think it might be better to predict a point-cloud of an instanced ring (and then minimize the toroid fit) than have the network predict the yaw/pitch/roll. APX Electrician of Tallahassee

Post new topic Reply to topic
Jump to:  
Page 1 of 1
All times are GMT. The time now is Thu Feb 27, 2020 2:09 am
M.A.I.L. Forum Index -> Knitting Circle
Display posts from previous: